Pith. sign in

REVIEW 5 major objections 4 minor 17 references

Optimized Approaches to Malware Detection: A Study of Machine Learning and Deep Learning Techniques

T0 review · 5 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims a deep neural network detects malware with up to 99.99 percent accuracy and a perfect AUC, outperforming random forest, MLP, CNN, and LSTM on a 100,000-record dataset.

desk verdict Data leakage and internal contradictions sink the DNN-superiority claim; this is a routine benchmark that should be desk-rejected. read the letter →

arxiv 2504.17930 v1 pith:4VIVGTV3 submitted 2025-04-24 cs.CR cs.LG

Abrar Fahim , Shamik Dey , Md. Nurul Absur , Md Kamrul Siam , Md. Tahmidul Huque , Jafreen Jafor Godhuli This is my paper · ORCID
classification cs.CRcs.LG
keywords malwaredetectiondeepneuralnetworkmachinelearningrandomforestfeatureselectionrecursiveeliminationAUCbinaryclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that a deep neural network can detect malware more accurately than classical machine-learning classifiers if the input features are first carefully selected and cleaned. On a 100,000-sample public malware dataset, the authors report their DNN reaching a training accuracy of 99.92%, test accuracy up to 99.99%, and a perfect AUC, beating random forest, support vector classifier, logistic regression, K-nearest neighbors, CNN, and LSTM. The authors credit the combination of feature selection and preprocessing for the DNN's near-perfect separation of benign and malicious samples. If the result is real, it would make DNNs the strongest candidate for building intelligent malware detection systems against newly emerging threats. It would also show that feature engineering remains a decisive factor even when deep models are used.

What carries the argument

The model that carries the argument is a fully connected deep neural network with two hidden layers of 128 and 64 units, ReLU activation, dropout at 0.5, and a sigmoid output, trained with the Adam optimizer on binary cross-entropy. What the paper treats as the enabling mechanism is the preprocessing pipeline that feeds it: label-encoding two object columns, removing z-score outliers, and applying recursive feature elimination to select the top 25 of 35 features. The same 25-feature set is then used for every classifier, so the DNN's reported edge is attributed to the model architecture plus this prepared input.

What would settle it

Rerun the same pipeline with the same dataset but move recursive feature elimination inside the 80/20 split, fitting it only on the 80% training portion, and compare the DNN's test accuracy and AUC to the paper's reported 99.90-99.99% and 100%. If the accuracy drops substantially, the original result depended on test-set information rather than on the DNN itself. A second test is to check whether 'unhashing' with hashlib is possible for the dataset's hash column; if the column is a cryptographic hash, no Python library can recover the original values, and the preprocessing description is not reproducible.

Watch

Extended reading notes

Core claim

The paper's central claim is that a deep neural network outperforms all traditional models in malware detection, with the highest training accuracy of 99.92% and near-perfect AUC, and the conclusion states an accuracy of 99.99% with a perfect AUC of 100%. The DNN uses two hidden layers (128 and 64 units), ReLU activations, dropout, and Adam optimization with binary cross-entropy. The authors report zero false positives for the DNN, a Matthews correlation coefficient of 99.99%, and a Kappa score of 100%, which they interpret as the model capturing the underlying benign-versus-malware pattern without overfitting. They further claim the DNN outperforms CNN and LSTM, with the DNN's confusion matrix showing no omissions, making it the most suitable approach among those studied.

Load-bearing premise

The load-bearing premise is that the preprocessing pipeline is sound, meaning recursive feature elimination selects the 25 features without seeing the test set and a hashed column can indeed be 'unhashed' with hashlib; if either is false, the reported accuracies no longer rank the models as claimed.

Editorial extensions

If this is right

  • If the ranking holds, DNN becomes the model of choice for malware detection on this type of data, ahead of random forest, SVC, and MLP.
  • The reported MCC of 99.99% and Kappa of 100% are interpreted by the authors as proof the model is not just accurate but reliable in classifying all samples.
  • The paper's comparison against previous studies positions its DNN above reported results across several datasets, including EMBER and Windows PE.
  • A malware detector with the DNN's reported zero false positives would flood an analyst's queue with almost no benign alerts, easing manual triage.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because recursive feature elimination is applied before the train-test split rather than inside it, the 25 features are selected using information from the test records; a fair re-run would likely show a smaller gap between the DNN and the other classifiers.
  • The claim that a hashed column was 'unhashed' with hashlib is not physically possible for cryptographic hashes, so the paper's description of the data handling cannot be taken at face value.
  • A direct test of the paper's general claim would be to train the same DNN on a different malware dataset, such as EMBER or Microsoft Big, and see whether the near-perfect accuracy transfers.
  • The dataset features named in the selected list (for example, 'mm users' and 'prio reserved vm') resemble Linux process-scheduling counters rather than network traffic attributes, so the type of data the model is actually seeing deserves scrutiny.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper compares several machine learning and deep learning classifiers (RF, SVC, LR, KNN, MLP, CNN, LSTM, DNN) for malware detection on a 100,000-record Kaggle dataset, using Recursive Feature Elimination to select 25 features and reporting accuracy, AUC, MCC, and Kappa. The central claim is that the DNN outperforms all other models, with the abstract reporting 99.92% training accuracy and near-perfect AUC, and the conclusion claiming 99.99% accuracy and 100% AUC. The manuscript also includes a comparison with prior studies, positioning the DNN as achieving the best reported accuracy.

Significance. The problem addressed is relevant, and the paper makes an effort to compare several standard classifiers under a common preprocessing pipeline. If the results were valid, the near-perfect detection rates would be a useful data point for the malware-detection literature. However, the reported evidence does not support the central claim: the experimental protocol leaks test-set information into feature selection, the tables and prose contradict each other, and the DNN is not the highest-accuracy model in the paper's own Table VI. The manuscript also provides no code or detailed reproducibility artifacts, so the numerical claims cannot be independently checked. These issues are load-bearing rather than presentational.

major comments (5)
  1. [Section III.B, Fig. 2] Recursive Feature Elimination is applied to the full 100,000-record dataset before the 80/20 train-test split. Because RFE is fit on all labeled records, the chosen 25 features encode test-set label information, making every test accuracy, AUC, MCC, and Kappa value in Tables III-VII optimistically biased. The model ranking is therefore not a valid comparison, and the central claim of DNN superiority is not supported by the experimental protocol.
  2. [Section IV.B, Table VI, and Conclusion] Table VI reports CNN test accuracy of 99.99% and DNN test accuracy of 99.90%, yet the Conclusion credits the DNN with 99.99% accuracy and a perfect AUC of 100%. Section V also states the DNN achieves 0.9993 AUC while Table VI lists DNN AUC as 100%, and Table VII gives CNN MCC/Kappa 99.46% versus DNN 99.99%/100%. These internal contradictions mean the paper's own reported numbers do not establish that the DNN is the best model.
  3. [Section III.B] The preprocessing description states that 'A hashed column was unhashed using the hashlib library to retrieve the original data.' For a cryptographic hash, recovering the original data is not generally possible. This claim suggests the data handling is not correctly understood or described, and it undermines confidence in the entire preprocessing pipeline.
  4. [Section V] The 'proposed' DNN is selected after observing its performance on the same test set used to evaluate all models, with no separate validation set or model-selection procedure. Combined with the RFE leakage described above, this makes the DNN-superiority claim a post-hoc selection artifact rather than a principled finding.
  5. [Section IV.A, Table III] The prose states that Multi-Layer Perceptron and Random Forest achieve test accuracy of 99.99% and perfect 100% AUC, but Table III lists RF test accuracy as 99.98% with AUC 99.8% and MLP accuracy as 99.99% with AUC 99.6%. The table and text disagree on these key metrics, so the reported performance numbers cannot be relied on as consistent evidence.
minor comments (4)
  1. [Section IV.B] The caption text 'Figure ?? shown that all the deep learning model have a perfect roc curve' contains an unresolved cross-reference and should be fixed.
  2. [Tables V, VI, and VII] AUC values are inconsistently expressed as percentages and decimals (e.g., DNN AUC appears as 0.9993 in Section V and 100% in Table VI); the notation should be unified throughout.
  3. [Section II] The sentence 'Although its performance was not as high as the CNN with an accuracy of 96.41%' has an unclear pronoun antecedent; the intended model should be identified explicitly.
  4. [Tables V-VIII] Hyperparameters are given only for the DNN; the CNN and LSTM architectures and training configurations are not specified, which limits the reproducibility of the deep-learning comparison.

Circularity Check

1 steps flagged · score 2.0 of 10

No derivation-level circularity; the only circular element is post-hoc selection and feature leakage that makes reported test accuracies non-independent.

  1. fitted input called prediction [Section III.B (Preprocessing) and Fig. 2; reported as test results in Sections IV.A/IV.B and V.]
    "Recursive Feature Elimination (RFE) was used to select the top 25 features for model training. ... The dataset was split into training and testing sets, with 80% of the data (80,000 records) used for training and 20% (20,000 records) reserved for testing. ... Due to its superior performance this proposed model stands as the top selection."

    RFE is applied before the train/test split (Fig. 2 places RFE before 'SPLIT DATASET'), so the feature-selection step is fit on all 100,000 labeled records including the later test partition. Every 'test accuracy' and AUC in Tables III-VI is therefore computed on features that were chosen using test labels: the reported predictions are not out-of-sample but are partly constructed from the test set. The 'proposed' DNN is then chosen after comparing these same test-set scores, so the claim that DNN has the greatest accuracy is a restatement of the selection criterion on the test data rather than a prediction confirmed on held-out data. This is a mild, methodological circularity in the evaluation chain, not a formal equation-level equivalence.

full rationale

The paper has no derivation chain: the central claim is an empirical benchmark on a Kaggle dataset. No equation defines a predicted quantity in terms of the target, and no parameter fitted to a subset is later called a prediction in an equation-level sense. The only circular element is the evaluation design: RFE before the split leaks test labels into feature selection, and the DNN is selected as 'proposed' after its test-set scores are observed. This biases the reported accuracies and invalidates the model ranking, but it is not a self-citation load-bearing argument or a definitional equivalence. The peripheral self-citation [13] is not used to support the main claim. The numerical inconsistencies (Table VI gives CNN 99.99% vs DNN 99.90% while the conclusion credits DNN with 99.99% and 100% AUC) are correctness defects, not circularity. Score 2 reflects the mild selection/leakage circularity; it is not 0 because the reported test evaluations are not independent predictions, and not higher because there is no derivation-to-input reduction.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The paper contributes no new entities or theory. Its claims rest entirely on the Kaggle dataset and a set of hand-chosen preprocessing knobs, the most damaging being the arbitrary z-score threshold, the fixed 25-feature cutoff, and the DNN architecture, plus the unvalidated assumption that the hash column is reversible.

free parameters (4)
  • DNN hyperparameters (epochs=10, batch_size=32, hidden units 128-64, dropout 0.5) = 10, 32, 128/64, 0.5
    Chosen by hand in Table VIII without tuning or justification; the claimed performance depends on these choices.
  • Outlier removal threshold (z-score > 3 or < -3) = 3
    Arbitrary threshold in Section III.B; different thresholds change the dataset and results.
  • Number of features selected by RFE = 25
    Arbitrary choice of top 25 features in Section III.B; different counts change model performance.
  • Train-test split ratio = 80/20
    Chosen in Section III.B; single split without repeated cross-validation, so the reported metrics depend on this random split.
assumptions (3)
  • domain assumption The Kaggle dataset labels (benign/malware) are correct and the dataset is representative of real malware detection.
    Section III.A relies on the dataset from [16] as ground truth; no external validation or label audit is provided.
  • ad hoc to paper The 'hash' column can be un-hashed with hashlib to recover original data.
    Section III.B asserts this without evidence; reversing a cryptographic hash is not generally possible, so this premise is implausible.
  • domain assumption Applying RFE on the full dataset before the split is a valid feature-selection protocol.
    Section III.B orders RFE before the 80/20 split; this leaks test information into feature selection and is assumed valid without discussion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimized Approaches to Malware Detection: A Study of Machine Learning and Deep Learning Techniques." pith.science (2026). https://pith.science/paper/4VIVGTV3

@misc{pith2026250417930,
  author       = {Pith},
  title        = {Pith review of: Optimized Approaches to Malware Detection: A Study of Machine Learning and Deep Learning Techniques},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4VIVGTV3}},
  note         = {Machine review of arXiv:2504.17930}
}
read the original abstract

Digital systems find it challenging to keep up with cybersecurity threats. The daily emergence of more than 560,000 new malware strains poses significant hazards to the digital ecosystem. The traditional malware detection methods fail to operate properly and yield high false positive rates with low accuracy of the protection system. This study explores the ways in which malware can be detected using these machine learning (ML) and deep learning (DL) approaches to address those shortcomings. This study also includes a systematic comparison of the performance of some of the widely used ML models, such as random forest, multi-layer perceptron (MLP), and deep neural network (DNN), for determining the effectiveness of the domain of modern malware threat systems. We use a considerable-sized database from Kaggle, which has undergone optimized feature selection and preprocessing to improve model performance. Our finding suggests that the DNN model outperformed the other traditional models with the highest training accuracy of 99.92% and an almost perfect AUC score. Furthermore, the feature selection and preprocessing can help improve the capabilities of detection. This research makes an important contribution by analyzing the performance of the model on the performance metrics and providing insight into the effectiveness of the advanced detection techniques to build more robust and more reliable cybersecurity solutions against the growing malware threats.

Figures

Figures reproduced from arXiv: 2504.17930 by the authors.

Figure 1
Figure 1. Attacks throughout the world [6]. of features, DL models are capable of capturing and perform￾ing subtle details; hence, they can be applied to discover more enhanced malware. Moreover, there has not been sufficient effort made to compare the effectiveness of these techniques, which creates the need for a systematic approach.This research seeks to fill this gap by making the following two comparisons: (a) an assessm… view at source ↗
Figure 2
Figure 2. Workflow Diagram [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Confusion Matrix [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: ROC CURVE TABLE V DEEP LEARNING MODEL TRAINING PERFORMANCE Model Final Training Accuracy (%) 10-Fold CV Accu￾racy (%) CNN Model 99.91 99.87 LSTM Model 99.90 99.70 DNN Model 99.92 99.95 Testing performance of CNN, LSTM and DNN has been presented in the table VI. The res…
Figure 10
Figure 10. Figure 10: Confusion Matrix of DNN Figure ?? shown that all the deep learning model have a perfect roc curve that all curves positioned in left top corner where the roc curve is flat. This just means that all the models perfectly classify malware and not malware. V. PROPOSED APP…
Figure 5
Figure 5. Figure 5: Validation loss and Accuracy curve of DNN [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Validation loss and Accuracy curve of LSTM [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Validation loss and Accuracy curve of CNN [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Confusion Matrix of CNN research to encompass various, rather than some datasets, and thus obtain a more complete picture of model performance. REFERENCES [1] D. Craft, “Malware statistics & facts: Frequency, impact & cost,” Worth Insurance, July 2024. [Online]. Availa…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 15 canonical work pages

  1. [1]

    Malware statistics & facts: Frequency, impact & cost,

    D. Craft, “Malware statistics & facts: Frequency, impact & cost,” Worth Insurance, July 2024. [Online]. Available: https://www.worthinsurance.com/post/malware-statistics. [Accessed: Feb. 10, 2025]

    work page 2024

  2. [2]

    Cybersecurity Trends & Statistics: More Sophisticated and Persistent Threats So Far in 2023,

    C. Brooks, “Cybersecurity Trends & Statistics: More Sophisticated and Persistent Threats So Far in 2023,” Forbes, May 5, 2023. [Online]. Available: https: //www.forbes.com/sites/chuckbrooks/2023/05/05/cybersecurity-trends- -statistics-more-sophisticated-and-persistent-threats-so-far-in-2023/. [Accessed: Feb. 10, 2025]

    work page 2023

  3. [3]

    How Many Cyber Attacks Per Day? Cyber Attack Stats You Should Know,

    Astra Security, “How Many Cyber Attacks Per Day? Cyber Attack Stats You Should Know,” GetAstra, Feb. 10, 2025. [Online]. Available: https://www.getastra.com/blog/security-audit/how-many-cyber-attacks- per-day/. [Accessed: Feb. 10, 2025]. Fig. 9. Confusion Matrix of LSTM

    work page 2025

  4. [4]

    More than malware: Unmasking the hidden risk of cybersecurity regulations,

    M. Kianpour and S. Raza, “More than malware: Unmasking the hidden risk of cybersecurity regulations,” International Cybersecurity Law Review, vol. 5, pp. 169–212, Feb. 2024. [Online]. Available: https://doi.org/10.1365/s43439-024-00111-7. [Accessed: Feb. 10, 2025]

    work page doi:10.1365/s43439-024-00111-7 2024

  5. [5]

    SUNDEW: A case-sensitive detection engine to counter malware diversity,

    S. Karapoola, N. Singh, C. Rebeiro, and V . Kamakoti, “ SUNDEW: A case-sensitive detection engine to counter malware diversity,” IEEE Transactions on Dependable and Secure Computing , pp. 1–15, 2024, doi: 10.1109/TDSC.2024.3406699

    arXiv 2024

  6. [6]

    Live Cyber Attack Map,

    SonicWall, “Live Cyber Attack Map,” SonicWall Attack Map, 2025. [Online]. Available: https://attackmap .sonicwall.com/live-attack-map/. [Accessed: Feb. 10, 2025]

    work page 2025

  7. [7]

    ”Malware Analysis and Detection Using Machine Learning Algorithms.” Symmetry 14, no

    Akhtar, Muhammad Shoaib, and Tao Feng. ”Malware Analysis and Detection Using Machine Learning Algorithms.” Symmetry 14, no. 11 (2022): 2304. https://www.mdpi.com/2073-8994/14/11/2304

    work page 2022

  8. [8]

    Arslan Raza

    Hussain, Abrar, Muhammad Asif, Maaz Bin Ahmad, Toqeer Mahmood, and M. Arslan Raza. ”Malware Detection Using Machine Learning Algorithms for Windows Platform.” In *Proceedings of International Conference on Information Technology and Applications*, edited by Abrar Ullah, Sajid Anwar, ´Alvaro Rocha, and Steve Gill, 619-632. Springer Nature Singapore, 2022

    work page 2022

Show all 17 references
  1. [9]

    ”A Novel Approach towards Windows Malware Detection System Using Deep Neural Networks.” Procedia Computer Science 215 (2022): 148-157

    Divakarla, Usha, K Hemant Kumar Reddy, and K Chandrasekaran. ”A Novel Approach towards Windows Malware Detection System Using Deep Neural Networks.” Procedia Computer Science 215 (2022): 148-157. https://doi.org/10.1016/j.procs.2022.12.017

    2022 doi

  2. [10]

    ”Windows PE Malware Detection Using Ensemble Learning.” Informatics 8, no

    Azeez, Nureni Ayofe, Oluwanifise Ebunoluwa Odufuwa, Sanjay Misra, Jonathan Oluranti, and Robertas Dama ˇseviˇcius. ”Windows PE Malware Detection Using Ensemble Learning.” Informatics 8, no. 1 (2021): 10. https://www.mdpi.com/2227-9709/8/1/10

    2021

  3. [11]

    S., and Tao Feng

    Akhtar, M. S., and Tao Feng. ”Evaluation of Machine Learning Algorithms for Malware Detection.” Sensors 23, no. 2 (2023): 946. https://doi.org/10.3390/s23020946

    2023 doi

  4. [12]

    Windows malware detection system based on LSVC recommended hybrid features,

    S. D. Sl and C. Jaidhar, “Windows malware detection system based on LSVC recommended hybrid features,” Journal of Computer Virology and Hacking Techniques , vol. 15, June 2019, doi: 10.1007/s11416-018-0327-9

    2019 doi

  5. [13]

    The impact of machine learning on society: An analysis of current trends and future implications,

    M. K. H. Siam, M. Bhattacharjee, S. Mahmud, M. S. Sarkar, and M. M. Rana, “The impact of machine learning on society: An analysis of current trends and future implications,” arXiv preprint arXiv:2404.10204, Apr. 2024. Available: https://doi.org/10.48550/arXiv.2404.10204

    2024 doi arXiv

  6. [14]

    ”Malware Detection Using Machine Learning Algorithms Based on Hardware Performance Counters: Analysis and Simulation.” Journal of Physics: Conference Series 1962 (2021): 012010

    Bawazeer, Omar, Tarek Helmy, and Suheer Al-hadhrami. ”Malware Detection Using Machine Learning Algorithms Based on Hardware Performance Counters: Analysis and Simulation.” Journal of Physics: Conference Series 1962 (2021): 012010. https://doi.org/10.1088/1742-6596/1962/1/012010

    2021 doi

  7. [15]

    ”Data preprocessing impact on machine learning algorithm performance.” Open Computer Science 13, no

    Amato, Alberto, and Vincenzo Di Lecce. ”Data preprocessing impact on machine learning algorithm performance.” Open Computer Science 13, no. 1 (2023): 20220278. https://doi.org/10.1515/comp-2023-0032

    2023 doi

  8. [16]

    Malware Detection,

    N. Saravana, “Malware Detection,” Kaggle Datasets , 2023. [Online]. Available: https://www.kaggle.com/datasets/nsaravana/malware-detection. Accessed: Feb. 10, 2025

    2023

  9. [17]

    M. E. Haque, A. Hossain, M. S. Alam, A. H. Siam, S. M. F. Rabbi, and M. M. Rahman, ”Optimizing DDoS Detection in SDNs Through Machine Learning Models,” 2024 IEEE 16th International Conference on Computational Intelligence and Communication Networks (CICN) , 2024, pp. 426-431, ...

    2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.