REVIEW 3 major objections 6 minor 34 references
Are Trees Really Green? A Detection Approach of IoT Malware Attacks
T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Energy-aware tuning of tree-based classifiers cuts inference energy by 60-90% while keeping detection MCC near 0.60, making on-device IoT intrusion detection viable.
desk verdict Useful green-ML result undermined by test-set selection in the evaluation protocol; the energy savings are real but the 'high performance' claim is not yet established. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a two-objective hyperparameter search: a single optimizer runs dozens of trials per model, maximizing MCC and minimizing the mean energy per test sample in micro-watt-hours, with energy measured by a profiling tool that reads the processor's power meter. The 'balanced' model is selected as the Pareto-front point closest to (0,1) in the energy-MCC plane. All models are trained on flow statistics—packet timings, sizes, and TCP flag counts computed bidirectionally and per direction—excluding IP addresses, ports, and payload, which keeps the analysis privacy-preserving and independent of encryption.
What would settle it
Run the same optimization pipeline with a genuinely held-out test set that is never touched during hyperparameter selection; if MCC drops well below 0.60 or the energy savings shrink when the model faces unseen traffic, the central claim would be refuted. Alternatively, repeat the experiment on a second labeled IoT traffic dataset and check whether the balanced single-tree still reaches about 0.60 MCC with 60-90% energy reduction.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that model selection for IoT malware detection should treat test-time energy as a first-class objective rather than an afterthought. A two-objective hyperparameter search over MCC and mean watt-hours per sample, with energy measured during inference, finds a balanced configuration on the Pareto front that matches or beats the default model's MCC (around 0.60) while reducing consumed resources by 60-90%. The single decision tree is the most efficient of the three algorithms; the ensembles deliver comparable MCC but at much higher energy. A further experiment removing port-scan flows, which are easily confused with benign traffic, pushes the single-tree MCC near 0.995 at about 2.35 micro-watt-hours per sample.
Load-bearing premise
The evaluation assumes that tuning the models on the test split and then measuring them on that same split gives an unbiased estimate of real performance on unseen IoT traffic, because no separate validation set is held out.
Editorial extensions
If this is right
- Default hyperparameter settings are far from energy-optimal for IoT detection; simply tuning for energy can cut inference cost by an order of magnitude.
- A single decision tree with balanced tuning reaches about 0.60 MCC at roughly 8 micro-watt-hours per sample, making it a stronger on-device candidate than the ensemble models.
- Because the features are payload-free and encryption-agnostic, the same detector can be applied to encrypted IoT traffic without decryption.
- The near-perfect MCC after removing port-scan flows indicates that much of the remaining error is a labeling and feature-similarity problem rather than a model-capacity problem.
- If the energy reductions transfer from the server testbed to real IoT hardware, on-premise ML-based intrusion detection becomes feasible on battery-powered devices.
Reading between the lines
- The energy figures were measured on a server processor, not on an actual IoT microcontroller; real-world savings on constrained hardware may differ, and the paper's central claim depends on that transfer.
- The paper's energy measurements depend on an energy-estimation tool that is mentioned in the text but has no corresponding entry in the reference list, so its reported accuracy is not verifiable from the paper.
- A stricter evaluation with a separate validation set or cross-validation would tell whether the reported MCC and energy values are predictive or are partly fitted to the test split.
- The same tuning recipe could be applied to other lightweight detectors, such as logistic regression or small neural networks, to see whether the 60-90% energy reduction is specific to tree ensembles.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes an energy-aware hyperparameter optimization methodology for tree-based IoT malware traffic classifiers (Decision Tree, Random Forest, Extra-Trees), using Optuna with a multi-objective function that minimizes per-sample inference energy (µWh, measured with Carbon Tracker) and maximizes Matthew's Correlation Coefficient. On the Aposemat IoT-23 dataset, the authors compare default, max-green, max-MCC, and balanced models, reporting that balanced models reach MCC around 0.60 while consuming 60–90% less energy per test sample than the default models. The paper also analyzes false negatives and argues that port-scan flows resemble legitimate traffic, reporting much higher MCC after removing port-scan flows from the evaluation.
Significance. Strengths: the paper addresses a relevant and under-studied problem (energy-aware model selection for on-premise IDS); uses a public dataset; uses privacy-preserving flow statistics; the energy reduction comparison is internally consistent because the same hardware and measurement tool are used for default and optimized models; and the selected hyperparameters (e.g., shallow trees, few estimators for max-green and balanced models) provide a credible mechanistic explanation for the reductions. If the performance claims were validated on truly unseen data, the result would be practically useful for resource-constrained deployments and would complement existing green-ML literature. The main limitation is that the reported MCC values may reflect selection on the test set rather than predictive performance; the post-hoc port-scan removal also changes the task. These issues are fixable with a proper validation protocol.
major comments (3)
- [Section 3.2–3.3 and Table 5] Section 3.2 creates only an 80-20 train/test split, and Section 3.3 states that 'Model optimization was performed based on energy consumption and performance during the testing phase.' As written, the same 20% test split appears to be used both for Optuna model selection and for reporting the MCC and µWh values in Table 5; no separate validation set or nested cross-validation is described. The balanced-model MCC values (0.60–0.61) and the corresponding energy numbers are therefore fitted selection outcomes, not unbiased estimates on unseen traffic, and the abstract's claim that 'models maintain high performance' is not yet supported. Please report results from a held-out validation set (or an inner CV loop) that is not used during the Optuna trials.
- [Section 4.4 and Figure 3] The error analysis removes the entire port-scan class from the dataset, which is the largest portion of the malicious traffic (over 10.9 million of roughly 11.7 million malicious flows in Table 1), and then reports MCC near 0.995 for the balanced single-tree model on the remaining data. This is a different and much easier classification task, and it does not support the paper's detection claim for real IoT traffic, which includes port scanning. Moreover, Section 4.4 explicitly states that the authors 'did not optimize the hyperparameters... as we expected it to have low accuracy,' so the high post-removal numbers are not produced by the proposed optimization methodology. The post-hoc removal should be presented only as an illustration of dataset bias, and the abstract and conclusion should not rely on it as evidence of maintained performance.
- [Section 4.1 and Section 5] All experiments are performed on a server, and Section 5 concedes that no constrained device was used. The paper's concluding claim that ML-based IDS 'are suitable for running on on-premise devices' and the abstract's suggestion of suitability for resource-constrained devices go beyond the evidence, because µWh measured on an Intel i9 server cannot be transferred to low-power IoT hardware without an execution model or actual measurements. Please either temper the conclusion to state that per-inference energy is low in absolute terms, or provide measurements (or validated energy models) on constrained hardware.
minor comments (6)
- [Section 4.2] The text states that the default models reach 'about 99% balanced accuracy,' but Table 5 reports balanced accuracy values of approximately 94.7%; please correct the inconsistency.
- [Section 4.4] The sentence 'The latter is expected because the less are the flows, the less complex the model is' is not clearly correct, since µWh is reported per test sample and the hyperparameters were not re-optimized after removal; please clarify the mechanism by which removing flows reduces per-sample energy.
- [Table 5 and Figure 3] The axis label in Figure 3 reads 'Wh' while the text and Table 5 use 'µWh'; please make the units consistent.
- [General] There are several typographical errors, including 'Secion' in Section 3.3 and 'Miraii' in the Introduction; please proofread the manuscript.
- [References] Reference [26] (the IoT-23 dataset) is missing publication venue and year information; please complete the bibliographic entry.
- [Abstract and Section 4.2] The abstract refers to 'detection accuracy,' but the paper's primary metric is MCC; please align the terminology with the evaluation metrics.
Circularity Check
Reported MCC values are Optuna objectives evaluated on the test split used for selection, so the 'maintain high performance' claim is a fitted result.
-
fitted input called prediction
[Section 3.2 and Section 3.3; reported in Table 5 and Section 4.3]
"Once we built our dataset, we randomly split it into a training and a test set following an 80-20 ratio. ... Model optimization was performed based on energy consumption and performance during the testing phase. ... we asked optuna to perform 64 iterations ... The optimizations aimed to maximize the MCC and minimize the energy footprint during the inference phase."
The same 20% split is used both for Optuna's objective (MCC and energy) and for the final Table 5 values. Selecting hyperparameters to maximize MCC and minimize energy on the test split, then reporting the MCC and energy of the selected model on that same split, makes the reported values selection optima rather than unbiased estimates for unseen traffic. The balanced model is specifically chosen as the point geometrically closest to (0,1) in that test-set Pareto front, so its headline 0.60 MCC and 60-90% energy reduction are constructed by the selection rule.
full rationale
The paper's energy-reduction comparison is credible and self-contained: default and optimized configurations are measured on the same hardware, and the direction of the effect is plausible from the chosen hyperparameters. However, the performance half of the central claim is statistically forced. Section 3.2 creates only an 80-20 train/test split, and Section 3.3 states that optimization is performed 'based on energy consumption and performance during the testing phase.' No separate validation set is described, so the test split is used both to select hyperparameters and to compute the reported MCC values. The balanced model is then selected as the Pareto-front point closest to (0,1), meaning the reported MCC and energy are the very objectives being optimized. This is a fitted-input-called-prediction pattern for the performance claim, not a circularity based on self-citation; the paper contains no load-bearing self-citation chain or imported uniqueness theorem. Weighing the independent energy result against the non-independent MCC result, the overall circularity score is 6: one of the paper's two headline predictions reduces by construction, while the other retains independent content.
Assumptions & free parameters
free parameters (3)
- single-tree balanced hyperparameters =
max_depth=13, min_leaf=5, min_split=13
- random forest balanced hyperparameters =
max_depth=17, min_leaf=6, min_split=20, max_features=7, estimators=18
- extra-trees balanced hyperparameters =
max_depth=14, min_leaf=2, min_split=18, max_features=24, estimators=204
assumptions (4)
- domain assumption The IoT-23 dataset is representative of realistic IoT malware traffic.
- domain assumption Carbon Tracker's RAPL-based estimates of CPU and RAM energy are accurate enough to compare model variants.
- domain assumption Energy measured on the server used for training reflects the energy a constrained IoT device would consume.
- ad hoc to paper Port-scan flows may be removed for error analysis because they resemble legitimate flows.
Cite this review
Pith. "Pith review of Are Trees Really Green? A Detection Approach of IoT Malware Attacks." pith.science (2026). https://pith.science/paper/53NOPGP7
@misc{pith2026250607836,
author = {Pith},
title = {Pith review of: Are Trees Really Green? A Detection Approach of IoT Malware Attacks},
year = {2026},
howpublished = {\url{https://pith.science/paper/53NOPGP7}},
note = {Machine review of arXiv:2506.07836}
}
read the original abstract
Nowadays, the Internet of Things (IoT) is widely employed, and its usage is growing exponentially because it facilitates remote monitoring, predictive maintenance, and data-driven decision making, especially in the healthcare and industrial sectors. However, IoT devices remain vulnerable due to their resource constraints and difficulty in applying security patches. Consequently, various cybersecurity attacks are reported daily, such as Denial of Service, particularly in IoT-driven solutions. Most attack detection methodologies are based on Machine Learning (ML) techniques, which can detect attack patterns. However, the focus is more on identification rather than considering the impact of ML algorithms on computational resources. This paper proposes a green methodology to identify IoT malware networking attacks based on flow privacy-preserving statistical features. In particular, the hyperparameters of three tree-based models -- Decision Trees, Random Forest and Extra-Trees -- are optimized based on energy consumption and test-time performance in terms of Matthew's Correlation Coefficient. Our results show that models maintain high performance and detection accuracy while consistently reducing power usage in terms of watt-hours (Wh). This suggests that on-premise ML-based Intrusion Detection Systems are suitable for IoT and other resource-constrained devices.
Figures
Reference graph
Works this paper leans on
-
[1]
What is the mirai botnet? https://www.cloudflare.com/it- it/learning/ddos/glossary/mirai-botnet/, 2025
work page 2025
-
[2]
The internet of things: A survey
Luigi Atzori, Antonio Iera, and Giacomo Morabito. The internet of things: A survey. Computer Networks, 54(15):2787–2805, 2010
work page 2010
-
[3]
Maldist: From encrypted traffic classification to malware traffic detection and classification
Ofek Bader, Adi Lichy, Chen Hajaj, Ran Dubin, and Amit Dvir. Maldist: From encrypted traffic classification to malware traffic detection and classification. In 2022 IEEE 19th Annual Consumer Communications & Networking Conference (CCNC), pages 527–533, 2022
work page 2022
-
[4]
Extending c2 traffic detection methodologies: From tls 1.2 to tls 1.3-enabled mal- ware
Diogo Barradas, Carlos Novo, Bernardo Portela, Sofia Romeiro, and Nuno Santos. Extending c2 traffic detection methodologies: From tls 1.2 to tls 1.3-enabled mal- ware. InProceedings of the 27th International Symposium on Research in Attacks, Intrusions and Defenses, RAID ’24, New York, NY, USA, 2024. Association for Computing Machinery
work page 2024
-
[5]
Disclosure: detecting botnet command and control servers through large-scale netflow analysis
Leyla Bilge, Davide Balzarotti, William Robertson, Engin Kirda, and Christo- pher Kruegel. Disclosure: detecting botnet command and control servers through large-scale netflow analysis. InProceedings of the 28th Annual Computer Security Applications Conference, ACSAC ’12, page 129–138, New York, NY, USA, 2012. Association for Computing Machinery
work page 2012
-
[6]
Semen Budennyy, Vladimir Lazarev, Nikita Zakharenko, Alexey Korovin, Olga Plosskaya, Denis Dimitrov, Vladimir Arkhipkin, Ivan Oseledets, Ivan Barsola, Ilya Egorov,AleksandraKosterina,andLeonidZhukov. eco2ai:Carbonemissionstrack- ing of machine learning models as the first step towards sustainable ai.Doklady Mathematics, 2022
work page 2022
-
[7]
’hide ’n seek’ botnet uses peer-to-peer infrastructure to compro- mise iot devices
Trend Business. ’hide ’n seek’ botnet uses peer-to-peer infrastructure to compro- mise iot devices. https://www.trendmicro.com/vinfo/us/security/news/internet- of-things/-hide-n-seek-botnet-uses-peer-to-peer-infrastructure-to-compromise-iot- devices, 2018
work page 2018
-
[8]
Daniele Canavese, Leonardo Regano, Cataldo Basile, Gabriele Ciravegna, and An- tonio Lioy. Encryption-agnostic classifiers of traffic originators and their applica- tion to anomaly detection.Computers & Electrical Engineering, 97:107621, 2022
work page 2022
Show all 34 references
-
[9]
Fairandgreenhyperpa- rameter optimization via multi-objective and multiple information source bayesian optimization
AntonioCandelieri,AndreaPonti,andFrancescoArchetti. Fairandgreenhyperpa- rameter optimization via multi-objective and multiple information source bayesian optimization. Machine Learning, 113, 2024
2024
-
[10]
Buchanan
Andrew Churcher, Rehmat Ullah, Jawad Ahmad, Sadaqat ur Rehman, Fawad Ma- sood, Mandar Gogate, Fehaid Alqahtani, Boubakr Nour, and William J. Buchanan. An experimental analysis of attack classification using machine learning in iot net- works. Sensors, 21(2), 2021
2021
-
[11]
C2miner: Tricking iot malware into revealing live command & control servers
Ali Davanian, Michail Faloutsos, and Martina Lindorfer. C2miner: Tricking iot malware into revealing live command & control servers. InProceedings of the 19th ACM Asia Conference on Computer and Communications Security, ASIA CCS ’24, New York, NY, USA, 2024. Association for Co...
2024
-
[12]
How to measure energy consumption in machine learning algo- rithms
Eva García-Martín, Niklas Lavesson, Håkan Grahn, Emiliano Casalicchio, and Veselka Boeva. How to measure energy consumption in machine learning algo- rithms. In ECML PKDD 2018 Workshops, pages 243–255, Cham, 2019. Springer International Publishing
2018
-
[13]
Estimation of energy consumption in machine learning.Journal of Parallel and Distributed Computing, 134:75–88, 2019
Eva García-Martín, Crefeda Faviola Rodrigues, Graham Riley, and Håkan Grahn. Estimation of energy consumption in machine learning.Journal of Parallel and Distributed Computing, 134:75–88, 2019. Green Machine Learning for IoT Detection 17
2019
-
[14]
Extremely randomized trees
Pierre Geurst, Damien Ernst, and Louis Wehenkel. Extremely randomized trees. Machine Learning, 63, 2006
2006
-
[15]
Long, Bui D
Truong Thu Huong, Ta Phuong Bac, Dao M. Long, Bui D. Thang, Nguyen T. Binh, Tran D. Luong, and Tran Kim Phuc. Lockedge: Low-complexity cyberattack detection in iot edge computing.IEEE Access, 9:29696–29710, 2021
2021
-
[16]
Iacovos Ioannou, Prabagarane Nagaradjane, Pelin Angin, Palaniappan Balasub- ramanian, Karthick Jeyagopal Kavitha, Palani Murugan, and Vasos Vassiliou. Gemlids-miot: A green effective machine learning intrusion detection system based on federated learning for medical iot networ...
2024
-
[17]
Flow- guard: An intelligent edge defense mechanism against iot ddos attacks
Yizhen Jia, Fangtian Zhong, Arwa Alrawais, Bei Gong, and Xiuzhen Cheng. Flow- guard: An intelligent edge defense mechanism against iot ddos attacks. IEEE Internet of Things Journal, 7(10):9552–9562, 2020
2020
-
[18]
Iot network traffic classification using machine learning algorithms: An experimental analysis
Rakesh Kumar, Mayank Swarnkar, Gaurav Singal, and Neeraj Kumar. Iot network traffic classification using machine learning algorithms: An experimental analysis. IEEE Internet of Things Journal, 9(2):989–1008, 2022
2022
-
[19]
New machine learning algorithm: Ran- dom forest
Yanli Liu, Yourong Wang, and Jian Zhang. New machine learning algorithm: Ran- dom forest. In Baoxiang Liu, Maode Ma, and Jincai Chang, editors,Information Computing and Applications, pages 246–252, Berlin, Heidelberg, 2012. Springer Berlin Heidelberg
2012
-
[20]
Keertikumar M., Shubham M., and R.M. Banakar. Evolution of iot in smart vehicles: An overview. In2015 International Conference on Green Computing and Internet of Things (ICGCIoT), pages 804–809, 2015
2015
-
[21]
Iot: Internet of threats? a survey of practical security vulnerabilities in real iot devices.IEEE Internet of Things Journal, 6(5):8182–8201, 2019
Francesca Meneghello, Matteo Calore, Daniel Zucchetto, Michele Polese, and An- drea Zanella. Iot: Internet of threats? a survey of practical security vulnerabilities in real iot devices.IEEE Internet of Things Journal, 6(5):8182–8201, 2019
2019
-
[22]
Threats of internet-of- thing on environmental sustainability by e-waste.Sustainability, 2022
Fathi Batoul Modarress, Alexander Ansari, and Ansari Al. Threats of internet-of- thing on environmental sustainability by e-waste.Sustainability, 2022
2022
-
[23]
Jurcut, and Mari- anne A
Basem Ibrahim Mukhtar, Mahmoud Said Elsayed, Anca D. Jurcut, and Mari- anne A. Azer. Iot vulnerabilities and attacks: Silex malware case study.Symmetry, 15(11), 2023
2023
-
[24]
Overview of use of decision tree algorithms in machine learning
ArundhatiNavada,AamirNizamAnsari,SiddharthPatil,andBalwantA.Sonkam- ble. Overview of use of decision tree algorithms in machine learning. In2011 IEEE Control and System Graduate Research Colloquium, pages 37–42, 2011
2011
-
[25]
Landscape of iot security.Computer Science Review, 44:100467, 2022
Eryk Schiller, Andy Aidoo, Jara Fuhrer, Jonathan Stahl, Michael Ziörjen, and Burkhard Stiller. Landscape of iot security.Computer Science Review, 44:100467, 2022
2022
-
[26]
Iot-23: A labeled dataset with malicious and benign iot network traffic
Maria Jose Erquiaga Sebastian Garcia, Agustin Parmisano. Iot-23: A labeled dataset with malicious and benign iot network traffic
-
[27]
Selcuk Uluagac, and Vehbi Cagri Gun- gor
Nazli Tekin, Abbas Acar, Ahmet Aris, A. Selcuk Uluagac, and Vehbi Cagri Gun- gor. Energy consumption of on-device machine learning models for iot intrusion detection. Internet of Things, 21:100670, 2023
2023
-
[28]
Charles Edison Tripp, Jordan Perr-Sauer, Jamil Gafur, Amabarish Nag, Avi Purkayastha, Sagi Zisman, and Erik A. Bensen. Measuring the energy consumption and efficiency of deep neural networks: An empirical analysis and design recom- mendations, 2024
2024
-
[29]
Visualization of pareto front approximations in evo- lutionary multiobjective optimization: A critical review and the prosection method
Tea Tušar and Bogdan Filipič. Visualization of pareto front approximations in evo- lutionary multiobjective optimization: A critical review and the prosection method. IEEE Transactions on Evolutionary Computation, 19(2):225–245, 2015. 18 Sanna et al
2015
-
[30]
’torii’ breaks new ground for iot malware
Jai Vijayan. ’torii’ breaks new ground for iot malware. https://www.darkreading.com/cyberattacks-data-breaches/-torii-breaks-new- ground-for-iot-malware, 2018
2018
-
[31]
An evolutionary study of iot malware.IEEE Internet of Things Journal, 8(20):15422– 15440, 2021
Huanran Wang, Weizhe Zhang, Hui He, Peng Liu, Daniel Xiapu Luo, Yang Liu, Jiawei Jiang, Yan Li, Xing Zhang, Wenmao Liu, Runzi Zhang, and Xing Lan. An evolutionary study of iot malware.IEEE Internet of Things Journal, 8(20):15422– 15440, 2021
2021
-
[32]
Internet of things in industries: A survey
Li Da Xu, Wu He, and Shancang Li. Internet of things in industries: A survey. IEEE Transactions on Industrial Informatics, 10(4):2233–2243, 2014
2014
-
[33]
Uncov- ering energy-efficient practices in deep learning training: Preliminary steps towards green ai
Tim Yarally, Luıs Cruz, Daniel Feitosa, June Sallou, and Arie van Deursen. Uncov- ering energy-efficient practices in deep learning training: Preliminary steps towards green ai. In2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)...
2023
-
[34]
Yokoyama, Mariza Ferro, and Bruno Schulze
André M. Yokoyama, Mariza Ferro, and Bruno Schulze. A multi-objective hyper- parameter optimization for machine learning using genetic algorithms: A green ai centric approach. In Ana Cristina Bicharra Garcia, Mariza Ferro, and Julio Cesar Rodríguez Ribón, editors,Advances in A...
2022
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.