{"id":"b9ba1d52-dc3d-492c-92dd-4e2edee65ab7","arxiv_id":"2412.19928","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Fine-tuned BERT-family models outperform GPT-2.0 on three cyberbullying-related datasets, with DistilBERT most efficient, though the paper's specific numbers are unreliable.","lead":"This paper tests five language models, from BERT to GPT-2.0, on three social media datasets for cyberbullying detection, measuring accuracy, speed, memory, and energy. It reports that fine-tuned BERT and RoBERTa beat GPT-2.0 and DistilBERT is most efficient, but the results are internally inconsistent.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"TweetEval's label set and the abstract/table mismatches leave the central BERT 95% claim without a valid cyberbullying benchmark.","rationale":"The reader's weakest assumption identifies the validity and comparability of the three datasets, and Table III's TweetEval labels are indeed the soft spot. The paper's own metrics are binary, but the experiments are multiclass, and the abstract's numbers do not match any single Table V row. This is not a matter of consensus disagreement; it is an internal and definitional problem. The conclusion even concedes that the adapted methods achieve only 'moderate detection rates,' further undercutting the abstract's 95% figures. No formal verification or reproducible artifact is provided beyond a GitHub link, so the numbers must stand on the tables alone. Because the central claim is precisely the quantitative comparison, the reader's REJECT verdict is appropriate. A revision that replaces or correctly relabels TweetEval, reports per-class or macro-averaged metrics over multiple seeds, and reconciles abstract and table values could make the qualitative finding testable, but the current submission does not support its headline numbers.","tokens_in":20616,"tokens_out":3504,"duration_ms":35872,"concrete_test":"Download the exact TweetEval split used in the linked GitHub repository and inspect the label file. If the labels are hate/offensive/emotion/religion/spiritual with no non-cyberbullying class, remap them to cyberbullying versus non-cyberbullying (or drop TweetEval) and recompute the headline metrics. If the reported 95/95/95/95/5% figures cannot be reproduced under a defensible binary mapping, the central empirical claim collapses.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing premise is that the three datasets measure the same construct: cyberbullying versus non-cyberbullying. Section IV-A and Table III list TweetEval classes as hate, offensive, emotion, religion, and spiritual. These are not a coherent cyberbullying taxonomy, and no non-cyberbullying class is present. Yet the evaluation metrics in Eqs. 8-11 are defined for binary TP/TN cyberbullying classification, and Table V reports five-class results without stating any one-vs-rest mapping or aggregation rule. If TweetEval is actually an offensiveness/emotion benchmark rather than a cyberbullying dataset, then the 95% BERT and 96% RoBERTa accuracies in Table V do not support the paper's cyberbullying-detection claims. The internal inconsistency between the abstract and Table V compounds this: the abstract's headline numbers (95% accuracy/precision/recall/F1, 5% error, 0.053 s inference) do not appear together in any single row. BERT's 95% metrics and 5% error occur on TweetEval, while its 0.053 s inference time appears on Kaggle. Thus the central quantitative claim is not reproducible from the reported tables, and the qualitative claim that tuned BERT-family models outperform GPT-2.0 is only as strong as the validity of the underlying labels.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports an empirical benchmark of five transformer-based text classifiers—BERT, RoBERTa, XLNet, DistilBERT, and GPT-2.0—for cyberbullying detection. The authors use three datasets (Kaggle, IEEE Data Port, and TweetEval), fine-tune the models with a uniform protocol, and evaluate accuracy, precision, recall, F1, error rate, inference time, RAM usage, CPU/GPU usage, and energy consumption. The headline conclusion is that BERT offers the best balance between predictive performance and computational efficiency, that RoBERTa is the strongest pure-performance model, that DistilBERT is the most resource-efficient, and that GPT-2.0 consistently underperforms the fine-tuned encoder models. The paper closes with recommendations for deploying lightweight models in real-time, resource-constrained settings and with a plan for future work on mitigation and multilingual detection.","tokens_in":20867,"tokens_out":7820,"duration_ms":73599,"significance":"If the empirical results were reliable, the paper would be a useful practical comparison for deploying NLP models in cyberbullying moderation systems, particularly because it includes resource measurements (latency, memory, energy) that many detection-focused papers omit. The authors also provide a GitHub link for code and data, which supports reproducibility in principle, and they explicitly acknowledge in the conclusion that adapted models yield only moderate detection rates, a welcome note of caution. However, the significance is currently contingent on resolving major data-validity and internal-consistency problems; the TweetEval labeling issue and the non-reproducible abstract numbers mean the paper cannot yet be used as a reliable guide.","major_comments":[{"comment":"TweetEval's class set (hate, offensive, emotion, religion, spiritual) is not a cyberbullying taxonomy and includes no non-cyberbullying class, yet the evaluation metrics in Equations (8)–(11) are defined for binary TP/TN cyberbullying detection. Table V reports five-class results for all datasets without specifying any one-vs-rest mapping or macro/micro aggregation rule. Because the abstract's 95% accuracy is taken from the TweetEval row, the paper's central quantitative claim is not supported by a valid cyberbullying benchmark. Please replace TweetEval with a genuine cyberbullying dataset, or explicitly define the multi-class mapping and aggregation and revise all claims and the abstract accordingly.","section":"§IV-A, Table III, §III-B"},{"comment":"The abstract's headline bundle (Accuracy 95%, Precision 95%, Recall 95%, F1 95%, Error Rate 5%, Inference Time 0.053 s, RAM 35.28 MB) does not correspond to any single configuration in the tables. In Table V, BERT's 95% metrics appear on TweetEval with an inference time of 0.095 s, while the 0.053 s inference time is the Kaggle result, and the 35.28 MB RAM value in Table VI is measured on Kaggle. The central stated result is therefore not reproducible from the reported data. Please report all numbers per dataset and, if a composite summary is intended, state the aggregation rule explicitly.","section":"Abstract vs. Tables V–VI"},{"comment":"The inference-time and energy-efficiency discussions contradict their own tables. In §V-F, the text says 'RoBERTa achieves the fastest inference time at 0.029 seconds, followed closely by GPT-2 with 0.012 seconds,' but Table V lists GPT-2.0 at 0.012 s and RoBERTa at 0.029 s, so GPT-2.0 is in fact the faster model. In §V-G, DistilBERT's Kaggle energy consumption of 0.0211 kWh is described as 'efficient,' yet Table VI shows BERT at 0.000263 kWh and RoBERTa at 0.000028 kWh, with only GPT-2.0 (0.0365 kWh) consuming more than DistilBERT. These contradictions directly undermine the resource-efficiency conclusions.","section":"§V-F, §V-G, Tables V–VI"},{"comment":"The implementation protocol is internally inconsistent and under-specified. Section IV-C states that all five classifiers used ReLU, cross-entropy loss, Adam with learning rate 0.0005, batch size 32, 10 epochs, and an 80:20 split, while Section IV-D claims the methods were applied 'exactly as described by their original authors'; these two statements cannot both be true for standard BERT/RoBERTa fine-tuning practice. Additionally, Table VI reports model memory that varies implausibly with dataset (e.g., DistilBERT at 1.09 MB on Kaggle, 1.20 MB on IEEE Data Port, and 100.96 MB on TweetEval), suggesting that the resource-usage measurement protocol needs to be reported and validated. Without this, the paper's 'balance' claim cannot be quantitatively assessed.","section":"§IV-C, §IV-D, Table VI"}],"minor_comments":[{"comment":"Equation (13) divides the total prediction time by N, which gives an average per-instance inference time, but the notation Tstart and Tend suggests a single instance; please clarify the definition.","section":"§III-B, Eq. (13)"},{"comment":"The text and conclusion state that RoBERTa achieved the highest accuracy across all datasets, but Table V shows DistilBERT highest on IEEE Data Port (87% vs. 82%); please reconcile the narrative with the table.","section":"§V-A(4), Conclusion"},{"comment":"The TweetEval dataset is attributed to reference [20], a paper on GPT benchmarking, rather than to the actual TweetEval benchmark resource; the citation should be corrected.","section":"Table III"},{"comment":"The notation 'n = 5' is ambiguous because it could mean five classes or five samples; please specify that n denotes the number of classes and explain how the reported metrics aggregate over classes.","section":"Table V"},{"comment":"The first sentence contains a typo ('HE Cyberbullying.1') that should read 'The Cyberbullying...' or 'Cyberbullying...'.","section":"Introduction"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's core qualitative message—that fine-tuned encoder models outperform GPT-2.0 for cyberbullying detection—is likely defensible on the Kaggle and IEEE Data Port datasets, but the TweetEval label mismatch, the non-reproducible abstract numbers, and the table/text contradictions are load-bearing. If the authors are not prepared to replace TweetEval with a valid binary cyberbullying dataset and correct the tables and abstract, rejection would be the appropriate outcome; with those corrections, the paper could become a useful empirical benchmark."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core numbers in this paper do not survive contact with its own tables. The abstract presents BERT's 95% accuracy, 0.053s inference time, and 35.28MB RAM as a single coherent result, but those figures come from different datasets: accuracy from TweetEval, inference time from Kaggle. The prose then compounds the problem by calling GPT-2's 0.012s inference time slower than RoBERTa's 0.029s. That's not a rounding error; it's an internal contradiction.\n\nWhat the paper does well is broaden the evaluation beyond accuracy to include inference time, RAM, CPU/GPU usage, and energy consumption across five models and three datasets. The GitHub link for code and data is a concrete move toward reproducibility, and the qualitative conclusion that fine-tuned BERT-family models beat GPT-2.0 is consistent with prior work. If the quantitative reporting were fixed, the efficiency comparison could be useful to practitioners in low-resource settings.\n\nBut the load-bearing flaw is the TweetEval dataset. Table III lists its classes as hate, offensive, emotion, religion, and spiritual. That is not a cyberbullying taxonomy; there is no non-cyberbullying class, and the evaluation metrics in Section III are defined for binary TP/TN cyberbullying classification. Running five models on a dataset that does not measure the construct the paper claims to evaluate invalidates the TweetEval results. The novelty claim is also overstated: the paper's own references include prior comparisons of GPT models and fine-tuned transformers on hate speech and cyberbullying tasks, so \"first comprehensive LLM comparison\" does not hold.\n\nThere are no error bars or repeated runs, so the reported differences (e.g., BERT 94% vs RoBERTa 93% on Kaggle) are not statistically grounded. The paper is not incoherent in its narrative, but the mismatch between abstract, prose, and tables, combined with the dataset validity problem, means the central claims are not reproducible as stated. A revision with corrected reporting, a genuinely cyberbullying-labeled dataset, and some measure of variance could be resubmitted, but as it stands the paper is not ready for peer review.\n\nThis is not a paper I would cite or recommend sending to referees. It might serve as a cautionary example in a reading group about benchmark selection, but the evidential value is low.","headline":"The paper's own tables contradict its headline numbers, and the TweetEval dataset is not a valid cyberbullying benchmark, so the central BERT 95% claim is unsupported.","tokens_in":21449,"tokens_out":2209,"would_cite":false,"duration_ms":22462,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuned transformer classifiers, especially BERT, detect cyberbullying more accurately and efficiently than the generative GPT-2.0, making BERT a practical choice for real-time moderation.","keywords":["cyberbullying detection","text classification","large language models","fine-tuning","BERT","RoBERTa","computational efficiency","social media"],"falsifier":"Re-run all five models on a single consistently labeled cyberbullying benchmark (for example, only hate/offensive versus non-harmful classes, or a unified dataset with the same label definition), and check whether BERT still reaches ~95% accuracy and GPT-2.0 still trails by 10 or more percentage points; if the gap narrows or reverses, the central claim is falsified.","tokens_in":20403,"feed_emoji":"🛡️","tokens_out":10967,"duration_ms":87721,"temperature":0.7,"pith_summary":"This paper compares five text-classification models—BERT, RoBERTa, XLNet, DistilBERT, and GPT-2.0—on three social media datasets to see which detects cyberbullying well enough for real-time use. The authors report that fine-tuned encoder models consistently surpass the generative GPT-2.0: BERT achieves about 95% accuracy, precision, recall, and F1 while keeping inference time near 0.05 seconds and memory around 35 MB, and RoBERTa reaches 96% accuracy on the largest dataset. DistilBERT is the lightest and fastest, and GPT-2.0 underperforms on accuracy while drawing more energy. The paper's practical conclusion is that fine-tuned encoders, with BERT as the balanced default, are better suited for real-time cyberbullying detection than generative models.","feed_headline":"Fine-tuned BERT beats GPT-2 on cyberbullying detection","feed_subtitle":"Encoder-based fine-tuning outperforms GPT-2 in accuracy and resource use across three datasets.","key_machinery":"The central object is a family of transformer-based language models—BERT, RoBERTa, XLNet, DistilBERT, and GPT-2.0—adapted as cyberbullying classifiers by fine-tuning a classification head on top of each pretrained model. The comparison runs every model over the same three datasets and evaluates nine metrics spanning accuracy, precision, recall, F1, error rate, inference time, memory, CPU/GPU usage, and energy. The argument's mechanism is the trade-off between a model's pretraining objective and its deployment cost: bidirectional encoder models capture harmful-text context better than the unidirectional decoder GPT-2.0, while distillation trades a little accuracy for large gains in speed and memory.","core_discovery":"On the paper's own terms, the discovery is that adapting general text-classification transformers to cyberbullying detection works best with encoder-based models that are fine-tuned on the target data. BERT, RoBERTa, and XLNet lead the accuracy rankings, RoBERTa reaching 96% accuracy on TweetEval and BERT achieving 95% across metrics; GPT-2.0 trails behind with 74-85% accuracy. The paper further claims the choice among models is a trade-off: RoBERTa maximizes performance but consumes more memory, DistilBERT minimizes resource use and inference time but loses accuracy on some datasets, and BERT sits between them with 0.053-second inference, 35.28 MB RAM, and 0.000263 kWh energy, making it the recommended balanced option for real-time deployment.","pith_inferences":["A direct testable extension is to rerun the five models on a single benchmark where every label is a cyberbullying category; TweetEval's emotion, religion, and spiritual classes are not obviously cyberbullying, so the rankings could change under a stricter task definition.","If the balance result generalizes, fine-tuned BERT-family encoders could serve as a first-pass moderation filter, with generative models used for synthetic data augmentation or explanation rather than classification.","The energy and memory numbers suggest DistilBERT could run on low-power edge devices, but that remains to be verified outside the specific hardware used in the paper.","Because the datasets differ in label sets and class distributions, per-platform threshold tuning would likely be needed in practice, which the paper does not address."],"forward_implications":["Deploying BERT or RoBERTa as fine-tuned classifiers should give a social media platform 95–96% accuracy on cyberbullying detection with sub-0.1-second inference.","Generative models like GPT-2.0 are a poorer choice for text-classification moderation, since they trail on accuracy and consume more memory and energy on the tested benchmarks.","DistilBERT is the resource-efficient alternative, with inference times as low as 0.001 seconds and the lowest memory usage, at a modest accuracy cost.","The reported footprint of BERT (about 35 MB RAM and 0.4% CPU/GPU) supports running it as a continuous monitoring filter rather than a batch process."],"supporting_citations":[{"why":"Supplies the Kaggle cyberbullying dataset (47,693 tweets) used for training and testing all five models.","marker":"[37]"},{"why":"Supplies the IEEE Data Port dataset (2,141 comments) used to evaluate performance on smaller, multi-class cyberbullying types.","marker":"[38]"},{"why":"Supplies the TweetEval dataset and the prior benchmarking of GPT models as classifiers that the paper extends and compares against.","marker":"[20]"},{"why":"Defines the BERT model that the paper fine-tunes and reports as the balanced performer.","marker":"[29]"},{"why":"Defines the RoBERTa model that achieves the top accuracy (96%) on TweetEval.","marker":"[30]"},{"why":"Defines the XLNet model used as a strong alternative in the comparison.","marker":"[31]"},{"why":"Defines DistilBERT, the distilled model the paper finds most resource-efficient.","marker":"[32]"},{"why":"Defines GPT-2.0, the generative model the paper shows underperforming in classification.","marker":"[33]"}],"fun_headline_variants":["BERT: balanced pick for cyberbullying detection","Fine-tuned transformers outshine generative AI on cyberbullying","RoBERTa tops accuracy; BERT wins on efficiency for cyberbullying","GPT-2 lags in cyberbullying detection benchmark"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the three datasets all measure cyberbullying in a comparable way, yet TweetEval's labels include emotion, religion, and spiritual categories that are not obviously cyberbullying, so if that dataset tests a different task the reported accuracies do not support the paper's conclusions about cyberbullying detection.","fun_headline_variants_meta":{"raw":{"variants":["BERT: balanced pick for cyberbullying detection","Fine-tuned transformers outshine generative AI on cyberbullying","RoBERTa tops accuracy; BERT wins on efficiency for cyberbullying","GPT-2 lags in cyberbullying detection benchmark"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000169,"raw_usage":{"total_tokens":1277,"prompt_tokens":969,"completion_tokens":308,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":585,"completion_tokens_details":{"reasoning_tokens":238}},"tokens_in":585,"tokens_out":308,"duration_ms":3687,"temperature":1.0,"reasoning_tokens":238,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:46:12.096746+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run all five models on a single consistently labeled cyberbullying benchmark (for example, only hate/offensive versus non-harmful classes, or a unified dataset with the same label definition), and check whether BERT still reaches ~95% accuracy and GPT-2.0 still trails by 10 or more percentage points; if the gap narrows or reverses, the central claim is falsified.","supporting_citations":[{"cited_title":"Sosnet: A graph convolutional network approach to fine -grained cyberbullying detection,","cited_arxiv_id":null,"evidence_quote":"Supplies the Kaggle cyberbullying dataset (47,693 tweets) used for training and testing all five models."},{"cited_title":"Bullytype: Impro ving and advancing cyber bullying types detection framework based on transformers approach,","cited_arxiv_id":null,"evidence_quote":"Supplies the IEEE Data Port dataset (2,141 comments) used to evaluate performance on smaller, multi-class cyberbullying types."},{"cited_title":"Offensiveness, hate, emotion and GPT: Benchmarking GPT3.5 and GPT4 as classifiers on Twitter-specific datasets,","cited_arxiv_id":null,"evidence_quote":"Supplies the TweetEval dataset and the prior benchmarking of GPT models as classifiers that the paper extends and compares against."},{"cited_title":"Cyberbullying detection using pre-trained bert model,","cited_arxiv_id":null,"evidence_quote":"Defines the BERT model that the paper fine-tunes and reports as the balanced performer."},{"cited_title":"Robertanet: Enhanced roberta transformer based model for cyberbullying detection with glove features,","cited_arxiv_id":null,"evidence_quote":"Defines the RoBERTa model that achieves the top accuracy (96%) on TweetEval."},{"cited_title":"The use of a large language model for cyberbullying detection,","cited_arxiv_id":null,"evidence_quote":"Defines the XLNet model used as a strong alternative in the comparison."},{"cited_title":"Bias and cyberbullying detection and data generation using transformer artificial intelligence models and top large language models,","cited_arxiv_id":null,"evidence_quote":"Defines DistilBERT, the distilled model the paper finds most resource-efficient."},{"cited_title":"Enhancing aggression detection using gpt-2 based data balancing technique,","cited_arxiv_id":null,"evidence_quote":"Defines GPT-2.0, the generative model the paper shows underperforming in classification."}],"review_version":1}