pith. sign in

arxiv: 2604.26229 · v1 · submitted 2026-04-29 · 💻 cs.CL

Comparative Analysis of AutoML and BiLSTM Models for Cyberbullying Detection on Indonesian Instagram Comments

Raihana Adelia Putri , Aisyah Musfirah , Anggi Puspita Ningrum , Luluk Muthoharoh , Ardika Satria , Martin Clinton Tosima Manullang This is my paper

Pith reviewed 2026-05-07 13:35 UTC · model grok-4.3

classification 💻 cs.CL
keywords cyberbullying detectionIndonesian languageInstagram commentsBiLSTMattention mechanismlogistic regressiontext preprocessingsocial media analysis
0
0 comments X

The pith

BiLSTM with attention outperforms other models for cyberbullying detection in Indonesian Instagram comments, while logistic regression leads among machine learning options after tailored preprocessing.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper evaluates machine learning and deep learning approaches on a balanced set of 650 Indonesian Instagram comments labeled as bullying or non-bullying. It applies a preprocessing pipeline that handles slang, removes stopwords, and stems informal text before testing Naive Bayes, logistic regression, and support vector machines with TF-IDF features alongside BiLSTM and BiLSTM with Bahdanau attention. The work shows that the attention-enhanced BiLSTM reaches the highest overall accuracy while logistic regression performs best among the machine learning models. These comparisons matter because they illustrate how language-specific preparation can improve results and that simpler models can still serve well when resources are limited.

Core claim

BiLSTM with Bahdanau attention achieves the strongest performance for classifying the Indonesian comments, followed by other deep learning and machine learning variants, with the results underscoring the contribution of domain-specific preprocessing steps to effective detection.

What carries the argument

BiLSTM with Bahdanau attention mechanism applied to preprocessed informal Indonesian text using slang normalization, stopword removal, and stemming.

If this is right

  • Domain-specific preprocessing for informal text boosts accuracy across both machine learning and deep learning models.
  • Deep learning models capture contextual patterns in cyberbullying more effectively than traditional feature-based approaches.
  • Logistic regression remains a viable choice for resource-constrained settings without substantial loss in detection quality.
  • The comparative results support choosing model types based on deployment constraints rather than accuracy alone.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar preprocessing strategies could be tested on cyberbullying detection for other low-resource languages on social platforms.
  • A practical system might combine logistic regression for fast initial filtering with BiLSTM attention for reviewing uncertain cases.
  • The small dataset size indicates that expanding the collection with more varied comments would help confirm the generalizability of the findings.

Load-bearing premise

The 650-comment dataset is representative of real-world Indonesian Instagram comments and the binary bullying labels are accurate without significant annotation noise or selection bias.

What would settle it

Evaluating the same models on a larger, independently sourced collection of Indonesian Instagram comments with fresh annotations would reveal whether the performance ranking between BiLSTM with attention and logistic regression holds.

Figures

Figures reproduced from arXiv: 2604.26229 by Aisyah Musfirah, Anggi Puspita Ningrum, Ardika Satria, Luluk Muthoharoh, Martin Clinton Tosima Manullang, Raihana Adelia Putri.

Figure 1
Figure 1. Figure 1: Systematic Stages of the Machine Learning Path. Meanwhile, the deep learning path in view at source ↗
Figure 2
Figure 2. Figure 2: Stages of the Deep Learning Architecture. 5 Experiments 5.1 Computing Environment All experiments were executed on a system based on a Central Processing Unit (CPU) using the Python programming language ≤ 3.11. The primary libraries used include PyCaret for automated machine learning (Auto ML), as well as PyTorch and Transformers for deep learning model development. 5.2 Hyperparameter Settings Parameter se… view at source ↗
read the original abstract

This study compares machine learning and deep learning approaches for cyberbullying detection in Indonesian-language Instagram comments. Using a balanced dataset of 650 comments labeled as Bullying and Non-Bullying, the study evaluates Naive Bayes, Logistic Regression, and Support Vector Machine with TF-IDF features, as well as BiLSTM and BiLSTM with Bahdanau Attention. A preprocessing pipeline tailored to informal Indonesian text is applied, including slang normalization, stopword removal, and stemming. The results show that Logistic Regression performs best among the machine learning models, while BiLSTM with Attention achieves the strongest overall deep learning performance. The findings highlight the value of domain-specific preprocessing and show that although deep learning captures contextual patterns more effectively, machine learning remains a competitive option for resource-constrained deployments.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. This paper conducts a comparative analysis of machine learning models (Naive Bayes, Logistic Regression, and Support Vector Machine using TF-IDF features) and deep learning models (BiLSTM and BiLSTM with Bahdanau Attention) for detecting cyberbullying in Indonesian Instagram comments. It utilizes a balanced dataset of 650 comments, applies a domain-specific preprocessing pipeline involving slang normalization, stopword removal, and stemming, and reports that Logistic Regression outperforms other ML models while BiLSTM with Attention achieves the best overall performance, underscoring the benefits of tailored preprocessing and the trade-offs between deep learning's contextual understanding and machine learning's efficiency in resource-limited scenarios.

Significance. Should the empirical comparisons prove robust upon detailed validation, the work would offer valuable insights into model selection for cyberbullying detection in low-resource languages like Indonesian, potentially aiding in the development of moderation tools for platforms such as Instagram. The emphasis on preprocessing for informal text is a positive aspect. However, given the current lack of quantitative results and methodological details, the significance is limited as the findings cannot be reliably evaluated or replicated.

major comments (3)
  1. [Abstract] The abstract states comparative results and highlights preprocessing value but supplies no numerical metrics, no train/test split details, no statistical significance tests, no error bars, and no ablation on preprocessing steps, leaving the central performance claims without sufficient supporting evidence.
  2. [Methodology] With a dataset of only 650 balanced samples, BiLSTM models have far more parameters than samples; the manuscript does not report train/test split ratios, k-fold CV, regularization details, or statistical significance tests (e.g., McNemar or paired t-test on F1), so any ranking of BiLSTM+Attention as strongest overall or LR as best among ML may be an artifact of a single split or random initialization.
  3. [Dataset] The assumption that the binary labels (Bullying vs Non-Bullying) are accurate and consistent is unvalidated; no details on the labeling process, inter-annotator agreement, or checks for annotation noise/selection bias in Instagram comments are provided, directly undermining claims that deep learning 'captures contextual patterns more effectively' in a generalizable way.
minor comments (2)
  1. [Title] The title refers to 'AutoML' models, but the abstract and body describe specific classifiers (Naive Bayes, Logistic Regression, SVM) without mentioning any AutoML framework, hyperparameter search, or automated pipeline.
  2. The manuscript would benefit from additional references to prior work on Indonesian-language cyberbullying detection or standard benchmarks for social media text classification.

Simulated Author's Rebuttal

3 responses · 1 unresolved

Thank you for your thorough review and valuable feedback on our manuscript. We appreciate the opportunity to clarify and strengthen our work. We will address each of the major comments by providing additional details, performing necessary analyses where possible, and revising the manuscript to enhance its methodological rigor and transparency.

read point-by-point responses

  1. Referee: [Abstract] The abstract states comparative results and highlights preprocessing value but supplies no numerical metrics, no train/test split details, no statistical significance tests, no error bars, and no ablation on preprocessing steps, leaving the central performance claims without sufficient supporting evidence.

    Authors: We agree that the abstract would benefit from including specific performance metrics and methodological details. In the revised version, we will incorporate the F1-scores and accuracies for the best-performing models, specify the 80/20 train/test split with 5-fold cross-validation, and discuss the preprocessing steps' contributions based on our internal ablations. While we did not conduct formal statistical significance tests or include error bars in the original submission, we will add standard deviations from cross-validation and note the lack of significance testing as a limitation. This will provide better support for our claims. revision: yes

  2. Referee: [Methodology] With a dataset of only 650 balanced samples, BiLSTM models have far more parameters than samples; the manuscript does not report train/test split ratios, k-fold CV, regularization details, or statistical significance tests (e.g., McNemar or paired t-test on F1), so any ranking of BiLSTM+Attention as strongest overall or LR as best among ML may be an artifact of a single split or random initialization.

    Authors: The concern regarding the small dataset size relative to the complexity of BiLSTM models is valid. Our experiments used stratified 80/20 splits and 5-fold cross-validation to evaluate performance more robustly. We applied dropout (rate 0.5) and L2 regularization in the BiLSTM architectures to address overfitting. However, we acknowledge that without statistical tests like McNemar's test, the rankings could be influenced by variability. We will expand the methodology section with these details, report mean and std dev across folds, and add a dedicated limitations subsection discussing the risks of overfitting and the need for larger datasets in future work. revision: yes

  3. Referee: [Dataset] The assumption that the binary labels (Bullying vs Non-Bullying) are accurate and consistent is unvalidated; no details on the labeling process, inter-annotator agreement, or checks for annotation noise/selection bias in Instagram comments are provided, directly undermining claims that deep learning 'captures contextual patterns more effectively' in a generalizable way.

    Authors: We recognize the importance of validating the label quality. The comments were manually labeled by the research team using a predefined set of cyberbullying indicators tailored to Indonesian social media contexts, with disagreements resolved through discussion. To address the lack of details, we will include a full description of the labeling guidelines and process in the revised dataset section. However, inter-annotator agreement was not formally calculated at the time of annotation, which we will acknowledge as a limitation. We will also discuss potential biases in sampling Instagram comments and how the domain-specific preprocessing helps mitigate some noise. revision: partial

standing simulated objections not resolved
  • Formal inter-annotator agreement scores for the binary labels, which were not computed during the original data annotation process.

Circularity Check

0 steps flagged

No circularity: purely empirical model comparison on held-out data

full rationale

The paper conducts standard supervised classification experiments: domain-specific preprocessing of Indonesian Instagram comments, TF-IDF vectorization for classical ML models (Naive Bayes, Logistic Regression, SVM), and training of BiLSTM variants with/without attention. Performance is evaluated via standard metrics on a balanced 650-sample dataset. No equations, derivations, or predictions are claimed; results are direct empirical observations. No self-citations are load-bearing for any central claim, and no fitted parameters are relabeled as independent predictions. The derivation chain is empty of the circular patterns defined.

Axiom & Free-Parameter Ledger

2 free parameters · 2 axioms · 0 invented entities

The central claim rests on the quality and representativeness of a small manually labeled dataset plus standard assumptions about TF-IDF feature extraction and neural network training for sequence classification.

free parameters (2)
  • BiLSTM and attention hyperparameters
    Number of layers, hidden size, learning rate, and attention parameters are fitted during training on the 650 samples.
  • TF-IDF vectorizer settings
    Maximum features, n-gram range, and other vectorizer choices are chosen or tuned to the dataset.
axioms (2)
  • domain assumption Manual binary labels on the 650 comments accurately reflect cyberbullying without significant noise or bias.
    No inter-annotator agreement or labeling protocol details are provided.
  • domain assumption The chosen preprocessing pipeline (slang normalization, stopword removal, stemming) improves performance for informal Indonesian text.
    Applied without reported ablation or comparison to unprocessed text.

pith-pipeline@v0.9.0 · 5457 in / 1549 out tokens · 93018 ms · 2026-05-07T13:35:40.930874+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

14 extracted references · 14 canonical work pages

  1. [1]

    Hinduja and J

    S. Hinduja and J. W. Patchin. Cyberbullying: Identification, prevention, and response. Cyberbullying Research Center, 2018

    work page 2018

  2. [2]

    Cyberbullying statistics 2024 by exposure and social media

    ElectroIQ. Cyberbullying statistics 2024 by exposure and social media. 2024

    work page 2024

  3. [3]

    Cyberbullying in Indonesia

    UNICEF. Cyberbullying in Indonesia. 2024

    work page 2024

  4. [4]

    Thakur and C

    N. Thakur and C. Y . Han. Cyberbullying detection in social media using natural language processing.Computers, Materials & Continua, 2023

    work page 2023

  5. [5]

    Sheth et al

    V . Sheth et al. A comparative analysis of machine learning algorithms for text classification.Procedia Computer Science, vol. 215, pp. 422–431, 2022. 6

    work page 2022

  6. [6]

    Yao et al

    M. Yao et al. A review on deep-learning-based cyberbullying detection.Future Internet, vol. 15, no. 5, p. 179, 2023

    work page 2023

  7. [7]

    Hafizh Fattah, M

    M. Hafizh Fattah, M. Alfan Rosid, S. Aji, and Suprianto. Hybrid CNN-LSTM for Indonesian cyberbullying detection on social media X.Journal of Informatics and Telecommunication Engineering, vol. 9, no. 2, 2026

    work page 2026

  8. [8]

    I. A. Asqolani and E. B. Setiawan. A hybrid deep learning approach leveraging Word2Vec feature expansion for cyberbullying detection in Indonesian Twitter.Ingenierie Des Systemes d’Information, vol. 28, no. 4, pp. 887–895, 2023

    work page 2023

  9. [9]

    Alfina, R

    F. Alfina, R. Mulia, M. I. Fanany, and Y . Ekanata. Hate speech detection in the Indonesian language: A dataset and preliminary study. InProceedings of ICACSIS, pp. 233–237, 2017

    work page 2017

  10. [10]

    C. T. Rahman, H. Nurdini, and N. Nurhasanah. Cyberbullying bahasa Indonesia dataset. Kaggle, 2023

    work page 2023

  11. [11]

    Nugraha and P

    D. Nugraha and P. Astuti. Analisis sentimen cyberbullying pada sosial media Instagram menggunakan metode support vector machine.Information System for Educators and Professionals, vol. 8, no. 2, pp. 152–164, 2023

    work page 2023

  12. [12]

    M. A. S. Nasution and E. B. Setiawan. Enhancing cyberbullying detection on Indonesian Twitter: Leveraging FastText for feature expansion and hybrid approach applying CNN and BiLSTM.Revue d’Intelligence Artificielle, vol. 37, no. 4, pp. 929–936, 2023

    work page 2023

  13. [13]

    Alabdulwahab, M

    A. Alabdulwahab, M. A. Haq, and M. Alshehri. Cyberbullying detection using machine learning and deep learning. International Journal of Advanced Computer Science and Applications, vol. 14, no. 10, 2023

    work page 2023

  14. [14]

    Slamet, et al

    C. Slamet, et al. Automated text summarization for Indonesian article using vector space model.2018 International Conference on Information Technology Systems and Innovation (ICITSI), IEEE, 2018. 7

    work page 2018