Pith. sign in

REVIEW 2 major objections 24 references

A Tsetlin Machine detects PDF malware at 98 percent accuracy while explaining each decision through readable logical rules.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 04:06 UTC pith:I6PYJDAE

load-bearing objection Solid first application of Tsetlin Machines to PDF malware: competitive accuracy, real interpretability demos, and a new public dataset, tempered by single-corpus evaluation after heavy cleaning. the 2 major comments →

arxiv 2607.09290 v1 pith:I6PYJDAE submitted 2026-07-10 cs.CR cs.LG

Leveraging Interpretable Tsetlin Machine for PDF Malware Detection

Rahul Jaiswal This is my paper
classification cs.CR cs.LG
keywords CybersecurityMalware DetectionPortable Document FormatTsetlin MachineInterpretable Machine LearningStatic Analysis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper sets out to show that PDF malware can be spotted by a Tsetlin Machine that learns human-readable logical clauses from static file features, without ever opening or running the document. On the RIT-PDFMal-2026 collection, after cleaning and balancing, the model reaches 98.02 percent accuracy and a macro F1 near 96 percent, staying competitive with Random Forest and gradient-boosting baselines while finishing inference in a few microseconds. Because every prediction is a transparent vote among propositional clauses, an analyst can see which rules and features drove the benign or malicious label. The work matters because PDFs remain a major attack channel and most high-performing detectors stay black boxes that need approximate post-hoc explanations. Intrinsic interpretability therefore turns competitive detection numbers into something security teams can audit and trust in practice.

Core claim

A Tsetlin Machine trained on 42 static numerical features extracted from PDF files without execution classifies them as benign or malicious at 98.02 percent accuracy (macro precision 96.03 percent, recall 95.95 percent, F1 95.99 percent) on the held-out RIT-PDFMal-2026 test set, matching or closely trailing strong black-box classifiers while supplying intrinsic explanations through clause activations, class-vote totals, and signed feature contributions.

What carries the argument

The Tsetlin Machine: a rule-based learner that builds conjunctive clauses from binary literals and their negations, then aggregates signed clause votes (bounded by threshold T) to decide the class. Those clauses are the human-readable decision rules that make classification transparent.

Load-bearing premise

The 42 static numerical features, after discarding more than a third of samples as duplicates and undersampling the benign class, still capture the patterns that matter for real-world PDF malware.

What would settle it

Re-run the identical feature extraction, preprocessing, and TM training pipeline on a fresh, independently collected PDF corpus that keeps natural class balance and includes malware families arriving after the original collection window; a large drop below the reported 98 percent accuracy would falsify the generalization claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Security operators can inspect activated clauses and top feature contributions to audit why any given PDF was flagged or cleared.
  • Static analysis plus TM inference runs in roughly three microseconds per sample, supporting high-volume scanning.
  • Competitive accuracy is obtained without post-hoc explanation tools such as SHAP or LIME.
  • The same clause representation surfaces which structural PDF elements (JavaScript, Encrypt, Launch, XFA, etc.) dominate malicious patterns.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the model relies only on static structural counts, it can serve as a cheap first-stage filter before slower sandboxes or dynamic analysis.
  • The higher false-negative rate relative to false positives suggests some malware families remain under-covered by the learned clauses; expanding the clause budget or feature set is a direct next experiment.
  • Readable rules open a path for human-in-the-loop editing in which analysts disable or refine clauses that fire on known false positives.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 0 minor

Summary. The paper proposes a Tsetlin Machine (TM) framework for static PDF malware detection. It extracts 42 numerical structural features without executing files, preprocesses the RIT-PDFMal-2026 corpus (duplicate removal, train-only undersampling, min-max scaling, KBinsDiscretizer binarization), and trains a TM with 250 clauses to classify benign vs. malicious PDFs. On a held-out 20% test split the model reports 98.02% accuracy and 95.99% macro F1, competitive with Random Forest (98.28%) and superior to XGBoost/LightGBM, while offering lower inference latency than RF and intrinsic interpretability via class-vote scores, clause-activation heatmaps, and feature-contribution plots. The authors position the combination of competitive accuracy, efficiency, and built-in explainability as the main contribution for practical PDF malware detection.

Significance. If the reported accuracy and interpretability claims hold under realistic distributions, the work supplies a concrete, low-latency alternative to black-box ensembles for a high-volume attack vector. The TM’s propositional clauses and vote/feature visualizations are a genuine methodological advantage over post-hoc SHAP/LIME explanations commonly applied to PDF detectors. The experimental pipeline is transparent (stratified split, train-only undersampling, five-fold CV, class-wise metrics, confusion matrix, seven baselines). The main limitation is that all evidence rests on a single, heavily cleaned public corpus; external validation or multi-dataset results would be required before the practical-deployment claim can be considered established. Within those bounds the contribution is solid and of clear interest to the malware-detection community.

major comments (2)
  1. Section V-C1 and Tables II/IV: After discarding 8,966 duplicates (36.84% of the original 24,337 samples) only 2,222 malicious files remain. Random undersampling is then applied solely to the training split, so the absolute number of malicious test examples is small. No ablation restores the original class prior, re-inserts near-duplicates, or evaluates an external hold-out corpus. Consequently the headline 98.02% accuracy / 95.99% macro-F1 figures (and the learned clauses shown in Figs. 7–15) may be optimistic relative to live PDF streams; at minimum the paper should quantify sensitivity to these preprocessing choices or report results on an independent corpus.
  2. Section V-D / Table VI: The state-of-the-art comparison juxtaposes methods evaluated on Contagio, Evasive-2022 and RIT-PDFMal-2026. Because the datasets differ in collection period, feature sets and class balance, the claim of “competitive performance … with existing methods” is only weakly supported. Either re-evaluate the baselines on the same RIT-PDFMal-2026 split or clearly qualify the comparison as non-head-to-head.

Circularity Check

0 steps flagged

No circularity: purely empirical supervised classification with held-out test metrics; self-citations are unrelated and non-load-bearing.

full rationale

The paper's central claims (98.02% accuracy / 95.99% macro F1 on RIT-PDFMal-2026, competitive with RF, plus intrinsic TM interpretability via clauses/votes/feature contributions) are obtained by standard supervised training and evaluation: static feature extraction, duplicate removal, stratified train/test split, random undersampling only on train, min-max + KBinsDiscretizer binarization, TM training with fixed hyperparameters (clauses=250, T=15, s=5, 50 epochs), and measurement of accuracy/precision/recall/F1/confusion matrix/inference time on the untouched test set (Tables III–V, Figs. 5–6). There is no mathematical derivation, uniqueness theorem, or fitted constant that is later re-presented as a prediction. Self-citations ([7]–[9], [23], [24]) concern the author's prior radio-map transfer-learning and speech-quality work and do not underwrite any malware-detection equation, feature set, or performance number; the TM itself is cited to Granmo and the dataset to Alani. Preprocessing choices affect absolute numbers but do not create a by-construction identity between inputs and reported outputs. The evaluation is therefore self-contained against the stated benchmark.

Axiom & Free-Parameter Ledger

5 free parameters · 3 axioms · 0 invented entities

The central empirical claim rests on standard ML modeling choices (binary classification, static features, undersampling) plus a handful of free hyper-parameters of the Tsetlin Machine and the binarizer. No new physical entities or unproved mathematical axioms are introduced; the load-bearing assumptions are domain conventions of static malware analysis and the representativeness of the single public dataset after aggressive cleaning.

free parameters (5)
  • number of clauses = 250
    Set to 250; controls model capacity and is chosen empirically (Table III).
  • voting threshold T = 15
    Set to 15; constrains accumulated clause votes (Eq. 2, Table III).
  • specificity s = 5
    Set to 5; controls literal inclusion probability during training (Table III).
  • KBinsDiscretizer n_bins = 15
    Set to 15 with quantile strategy; converts continuous features to binary literals required by TM (Table III).
  • training epochs = 50
    Fixed at 50; convergence shown in Fig. 5 but not ablated.
axioms (3)
  • domain assumption Static numerical features extracted without execution are sufficient to discriminate malicious from benign PDFs at the reported accuracy.
    Invoked throughout Sections III–V; no dynamic analysis or sandbox execution is performed.
  • domain assumption Random undersampling of the majority class on the training split does not destroy the decision boundary needed for generalization to the original imbalanced test distribution.
    Applied in Section V-C1; test set remains imbalanced while train is forced balanced.
  • domain assumption Propositional clauses learned by the Tsetlin Machine constitute faithful, human-interpretable explanations of the model’s decisions.
    Core claim of intrinsic interpretability (Abstract, Section V-C3); no user study validates that analysts actually understand or trust the clauses.

pith-pipeline@v1.1.0-grok45 · 14256 in / 2816 out tokens · 33940 ms · 2026-07-13T04:06:46.228124+00:00 · methodology

0 comments
read the original abstract

In the digital era, Portable Document Format (PDF) is one of the most widely used file formats for storing and exchanging digital documents due to its platform independence and rich functionality. However, these same capabilities have also made PDF files an attractive attack vector for cyberattackers, who embed malicious code within seemingly legitimate documents to compromise target systems. This paper presents a novel interpretable Tsetlin Machine (TM)-based framework for PDF malware detection. The proposed framework extracts salient features from PDF documents through static analysis without executing the files and employs rule-based learning to accurately classify benign and malicious PDF documents. Numerical evaluation on the RIT-PDFMal-2026 dataset demonstrates that the proposed framework achieves competitive performance, attaining an accuracy of 98.02% compared with several ML classifiers and existing methods. Moreover, the proposed framework provides intrinsic interpretability by transparently explaining its classification decisions. The combination of competitive detection performance, computational efficiency, and intrinsic interpretability makes the proposed framework a promising solution for practical PDF malware detection.

Figures

Figures reproduced from arXiv: 2607.09290 by Rahul Jaiswal.

Figure 1
Figure 1. Figure 1: The PDF internal architecture. The widespread adoption of PDF documents and their advanced functionalities have made them an attractive target for cyberattackers. Features such as embedded objects and JavaScript can be exploited to deliver malicious payloads, making PDF files a common attack vector for malware dis￾tribution. Malicious PDFs can facilitate cyberattacks such as credential theft, spyware insta… view at source ↗
Figure 2
Figure 2. Figure 2: Proposed TM framework for PDF malware detection. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Class imbalance in the dataset. Benign (0) Malicious (1) Class 0 2000 4000 6000 8000 10000 Number of Samples 10519 1777 1777 1777 Before Undersampling After Undersampling [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Balanced training classes. 0 10 20 30 40 50 Epoch 0.92 0.93 0.94 0.95 0.96 0.97 0.98 Accuracy Training Accuracy Testing Accuracy [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Training and testing accuracy across epochs. [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Class-wise votes of a Benign sample. 0 25 50 75 100 125 150 175 200 225 Clause Index Benign (0) Malicious (1) Classes True Label: Benign (0) | Predicted Label: Benign (0) | Decision: Correct 0.0 0.2 0.4 0.6 0.8 1.0 [PITH_FULL_IMAGE:figures/full_fig_p005_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Clause activation heatmap of the same Benign sample. [PITH_FULL_IMAGE:figures/full_fig_p005_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Top ten features of the same Benign sample. [PITH_FULL_IMAGE:figures/full_fig_p005_9.png] view at source ↗
Figure 6
Figure 6. Figure 6: TM confusion matrix. performing RF classifier (98.28%) and higher than those of XGBoost (97.37%) and LightGBM (97.30%). Moreover, the TM attains macro-averaged precision, recall, and F1-score of 96.03%, 95.95%, and 95.99%, respectively, demonstrating balanced classification performance across both benign and malicious PDF classes. Although RF achieves marginally higher accuracy and F1-score, it requires a … view at source ↗
Figure 10
Figure 10. Figure 10: Feature-level contribution of the same Benign sample as in Fig. 7. [PITH_FULL_IMAGE:figures/full_fig_p006_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Feature-level contribution of the same Malicious sample as in Fig. 12. [PITH_FULL_IMAGE:figures/full_fig_p006_11.png] view at source ↗
Figure 13
Figure 13. Figure 13: Clause activation heatmap of same Malicious sample. [PITH_FULL_IMAGE:figures/full_fig_p006_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Class-wise votes of another Benign sample. [PITH_FULL_IMAGE:figures/full_fig_p007_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Clause activation heatmap of the sample as in Fig. 14. [PITH_FULL_IMAGE:figures/full_fig_p007_15.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 1 linked inside Pith

  1. [1]

    Digital files presence in the world, 2025,

    CloudFiles, “Digital files presence in the world, 2025,” Accessed on July 05, 2026. [Online]. Available: https://www.cloudfiles.io/blog/how -many-files-are-there-in-the-world

    2025

  2. [2]

    Malware Detection in PDF and Office Documents: A Survey,

    P. Singh, S. Tapaswi, and S. Gupta, “Malware Detection in PDF and Office Documents: A Survey,”Information Security Journal: A Global Perspective, vol. 29, no. 3, pp. 134–153, 2020

    2020

  3. [3]

    Malicious PDF Attacks on Microsoft Windows, 2026,

    R. Informatica, “Malicious PDF Attacks on Microsoft Windows, 2026,” Accessed on July 05, 2026. [Online]. Available: https: //reisinformatica.com/malicious-pdf-attacks-on-microsoft-windows-202 6-protection-guide-for-canadian-businesses/

    2026

  4. [4]

    V APD: An Anomaly Detection Model for PDF Malware Forensics with Adversarial Robust- ness,

    S. Liu, J. Ming, Y . Zhou, J. Fu, and G. Peng, “V APD: An Anomaly Detection Model for PDF Malware Forensics with Adversarial Robust- ness,” in34th USENIX Security Symposium, 2025, pp. 4759–4778

    2025

  5. [5]

    PDF Malware Detection Based on Optimizable Decision Trees,

    Q. Abu Al-Haija and H. Qattous, “PDF Malware Detection Based on Optimizable Decision Trees,”Electronics, vol. 11, no. 19, p. 3142, 2022

    2022

  6. [6]

    Leveraging Machine Learning-Based PDF Malware Detection in Snort,

    F. Chbib and R. Khatoun, “Leveraging Machine Learning-Based PDF Malware Detection in Snort,” inInt. Conference on Electrical, Computer, Communications and Mechatronics Engineering. IEEE, 2024, pp. 1–6

    2024

  7. [7]

    Leveraging Transfer learning for Radio Map Estimation via Mixture of Experts,

    R. K. Jaiswal, M. Elnourani, S. Deshmukh, and B. Beferull-Lozano, “Leveraging Transfer learning for Radio Map Estimation via Mixture of Experts,”IEEE TCCN, vol. 12, pp. 846–863, 2025

    2025

  8. [8]

    Location-free Indoor Radio Map Estimation using Transfer learning,

    R. Jaiswal, M. Elnourani, S. Deshmukh, and B. Beferull-Lozano, “Location-free Indoor Radio Map Estimation using Transfer learning,” in97th Vehicular Technology Conference. IEEE, 2023, pp. 1–7

    2023

  9. [9]

    A Data-driven Transfer Learning Method for Indoor Radio Map Estima- tion,

    R. K. Jaiswal, M. Elnourani, S. Deshmukh, and B. Beferull-Lozano, “A Data-driven Transfer Learning Method for Indoor Radio Map Estima- tion,”IEEE TVT, vol. 75, no. 3, pp. 4261–4277, 2026

    2026

  10. [10]

    TransNet: Unseen Malware Variants Detection using Deep Transfer Learning,

    C. Rong, G. Gou, M. Cui, Z. Li, and L. Guo, “TransNet: Unseen Malware Variants Detection using Deep Transfer Learning,” inInt. Conf. on Security and Privacy in Communication Systems, 2020, pp. 84–101

    2020

  11. [11]

    A Unified Approach to Interpreting Model Predictions,

    S. M. Lundberg and S.-I. Lee, “A Unified Approach to Interpreting Model Predictions,” inNIPS, 2017, pp. 1–10

    2017

  12. [12]

    A Comprehensive Review of Tsetlin Machines: Concepts, Applications, Analysis, and the Future,

    S. Kundu, S. M. Mishra, G. Trivedi, and F. Merchant, “A Comprehensive Review of Tsetlin Machines: Concepts, Applications, Analysis, and the Future,”IEEE IoT Journal, vol. 13, no. 10, pp. 20 105–20 127, 2026

    2026

  13. [13]

    The Tsetlin Machine–A Game Theoretic Bandit Driven Approach to Optimal Pattern Recognition with Propositional Logic,

    O.-C. Granmo, “The Tsetlin Machine–A Game Theoretic Bandit Driven Approach to Optimal Pattern Recognition with Propositional Logic,” arXiv preprint arXiv:1804.01508, pp. 1–42, 2018

    Pith/arXiv arXiv 2018

  14. [14]

    Breiman, J

    L. Breiman, J. Friedman, R. A. Olshen, and C. J. Stone,Classification and Regression Trees. Chapman and Hall/CRC, 2017

    2017

  15. [15]

    Alpaydin,Introduction to Machine Learning

    E. Alpaydin,Introduction to Machine Learning. MIT press, 2020

    2020

  16. [16]

    D. W. Hosmer Jr, S. Lemeshow, and R. X. Sturdivant,Applied Logistic Regression. John Wiley & Sons, 2013

    2013

  17. [17]

    Xgboost: A Scalable Tree Boosting System,

    T. Chen and C. Guestrin, “Xgboost: A Scalable Tree Boosting System,” in22nd ACM SIGKDD International Conference on Knowledge Discov- ery and Data Mining, 2016, pp. 785–794

    2016

  18. [18]

    Lightgbm: A Highly Efficient Gradient Boosting Decision Tree,

    G. Ke, Q. Meng, and T. Finley, “Lightgbm: A Highly Efficient Gradient Boosting Decision Tree,” inNIPS, 2017, pp. 1–9

    2017

  19. [19]

    Handling Imbalanced Data: SMOTE vs Random Undersam- pling,

    S. Mishra, “Handling Imbalanced Data: SMOTE vs Random Undersam- pling,”International Research Journal of Engineering and Technology, vol. 4, no. 8, pp. 317–320, 2017

    2017

  20. [20]

    Bhagwat, M

    R. Bhagwat, M. Abdolahnejad, and M. Moocarme,Applied Deep Learn- ing with Keras: Solve Complex Real-life Problems with the Simplicity of Keras. Packt Publishing Ltd, 2019

    2019

  21. [21]

    RIT-PDFMal-2026: A Comprehensive Benchmark Dataset for PDF Malware Detection,

    M. M. Alani and E. Damiani, “RIT-PDFMal-2026: A Comprehensive Benchmark Dataset for PDF Malware Detection,”IEEE Access, vol. 14, pp. 97 841–97 855, 2026

    2026

  22. [22]

    Malicious Sample,

    VirusTotal, “Malicious Sample,” Accessed on July 05, 2026. [Online]. Available: https://www.virustotal.com/gui/home/upload

    2026

  23. [23]

    Performance Analysis of V oice Activity Detector in Pres- ence of Non-stationary Noise,

    R. Jaiswal, “Performance Analysis of V oice Activity Detector in Pres- ence of Non-stationary Noise,” in11th International Conf. on Robotics, Vision, Signal Processing and Power Applications, 2022, pp. 59–65

    2022

  24. [24]

    Non-intrusive Speech Quality Assess- ment using Context-aware Neural Networks,

    R. K. Jaiswal and R. K. Dubey, “Non-intrusive Speech Quality Assess- ment using Context-aware Neural Networks,”International Journal of Speech Technology, vol. 25, no. 4, pp. 947–965, 2022

    2022