Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Federated Isolation Forest for Efficient Anomaly Detection on Edge IoT Systems

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A federated isolation forest detects edge temperature anomalies with AUC-ROC 0.9996 while training in 153 KB of memory.

desk verdict PFLiForest is a genuine, honest port of FLiForest to MicroPython with real memory and AUC measurements, but the unspecified split aggregation and threshold-on-test-set leave the headline numbers softer than they look. read the letter →

arxiv 2506.05138 v2 pith:T6JR6X5K submitted 2025-06-05 cs.LG cs.DC

Pavle Vasiljevic , Milica Matic , Miroslav Popovic This is my paper
classification cs.LGcs.DC
keywords federatedlearningisolationforestanomalydetectioninternetofthingsedgecomputingtemperaturePTB-FLAunsupervised
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a federated isolation forest, called PFLiForest, can detect temperature anomalies on small edge IoT devices with near-centralized accuracy while keeping training memory small enough for a 256 KB microcontroller. The authors specialize the FLiForest algorithm on the PTB-FLA testbed, building isolation trees layer by layer: each client computes split values from its private data, the server averages them, and clients continue splitting locally. Across 40-run averaged experiments over tree depth, forest size, and training-data amount, the selected configuration (6, 25, 200) reaches AUC-ROC 0.9996 and AUC-PR 0.9728 at 153 KB of training memory. A comparison against the original iForest shows the federated version trails in AUC-PR but approaches the baseline as model size grows. If correct, this makes privacy-preserving, unsupervised anomaly detection practical on resource-constrained sensors.

What carries the argument

The load-bearing mechanism is layer-by-layer federated tree construction with averaged split aggregation. Instead of aggregating whole trees, each client privately computes its own candidate split values on the local partition, and the server aggregates them by averaging and broadcasts the single global split value; clients then partition their local data with that value and proceed to the next layer, with a queue-based iterative build and phase synchronization keeping all nodes at the same tree depth. Inference uses the standard isolation forest anomaly score $s(x,n)=2^{-E(h(x))/c(n)}$, with $c(n)=2(\ln(n-1)+0.577)-2(n-1)/n$, averaged over the forest.

What would settle it

Train the baseline iForest on exactly the same 200 temperature values used for point E and compare its AUC-PR with PFLiForest's 0.9728; if the matched-data iForest clearly beats the federated model, the claim that federation preserves detection quality fails. A second check is to run PFLiForest with two clients holding disjoint temperature ranges (e.g., 0–10°C and 30–40°C) and inspect whether the averaged split placed at the boundary misclassifies one client's normal range as anomalous.

Watch

Extended reading notes

Core claim

The central claim is that the layer-by-layer federated training protocol preserves enough of the isolation forest signal to be deployable on edge hardware. In PFLiForest, the client callback (Algorithm 3) computes candidate split points on private local data; the server callback (Algorithm 4) filters out resting clients and returns the average of the remaining clients' split values; clients then split their local partitions at that shared value and enqueue the children, continuing until depth or data-purity limits are reached and a phase-synchronization signal ends the tree. The resulting forest is evaluated on synthetic test sets built from real temperature readings with 10% injected anomalies. At the chosen operating point E (max_depth=6, num_trees=25, training data 200), PFLiForest obtains AUC-ROC 0.9996 and AUC-PR 0.9728 using about 153 KB during training, and at the F1-optimal threshold 0.8265 it reports 99.17% precision, 99.8% recall, and a false positive rate below 0.1%. Against the baseline iForest run with larger training sets, the federated model has lower AUC-PR but converges toward the baseline as model size grows, which the paper attributes to the privacy-preserving federated setting.

Load-bearing premise

The load-bearing premise is that averaging the split values computed independently by clients yields a single global isolation tree whose anomaly scores remain as reliable as a tree built on all data together.

Editorial extensions

If this is right

  • The selected model fits in the 256 KB RAM of the target Raspberry Pi Pico, using 153 KB during training.
  • Raw sensor readings never leave a client; only numeric split values are shared, so the scheme upholds federated privacy while still producing a shared model.
  • The unsupervised, threshold-based inference means a deployed device can be retrained on new temperature conditions without labeled anomalies.
  • At the F1-optimal threshold, 99.17% precision and 99.8% recall imply false alarms below 0.1%, a rate acceptable for safety and cold-chain monitoring.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: run the protocol with non-IID client temperature distributions (one sensor in a cold room, one near a heat source) and measure whether averaged splits drift from the globally optimal thresholds; the paper does not report how data is partitioned across clients.
  • The paper attributes the iForest performance gap to each client seeing less data, but it does not isolate the averaging operator's contribution; a matched-data comparison would show how much of the gap is federation cost versus data quantity.
  • Because the construction is layer-by-layer and communicates only scalars, it should transfer to other univariate telemetry, such as vibration amplitude or current draw, with the same memory footprint.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents PFLiForest, a specialization of the FLiForest federated isolation forest algorithm for the PTB-FLA/MPT-FLA frameworks, aimed at temperature anomaly detection on resource-constrained IoT devices. Tree construction is iterative and layer-by-layer: clients compute split values from their private data and the server aggregates them by averaging (Algorithms 1-4). The authors evaluate PFLiForest on a synthetic temperature dataset built by adding Gaussian noise and random anomalies to real sensor data, measuring AUC-ROC, AUC-PR, memory usage, and training time across four parameter configurations (points A-D). They then select point E=(6,25,200) as satisfying the 160 KB memory constraint and report AUC-ROC 0.9996 and AUC-PR 0.9728 at E, with 99.17% precision and 99.8% recall at a threshold chosen on the same test set. A comparison with centralized iForest is also reported, using doubled training-data sizes for iForest.

Significance. If the approach is valid, the contribution is a concrete demonstration that a federated isolation forest can run on MicroPython-class devices with a small memory footprint, which is practically useful for privacy-preserving IoT anomaly detection. The paper provides clear pseudocode for the PTB-FLA integration, measures memory and execution time, and acknowledges communication cost and performance trade-offs in the conclusion. However, the central algorithmic operation, averaging client-computed split values, is neither fully specified nor validated, and the evaluation protocol has selection-on-test-set issues; the high reported numbers should be read as upper bounds until these are addressed.

major comments (4)
  1. [II.B, Algorithms 3 and 4] The core aggregation step is not defined and not validated. Algorithm 3 (line 6) delegates split computation to client_process_layer(privateData), which is never specified in the paper, and Algorithm 4 (line 12) returns server_aggregate_layer(clientSplits), which the text describes only as 'averaging their values.' Since every split in every tree is the result of this average, the whole model depends on an operation whose statistical properties are unknown. In particular, iForest's anomaly score s(x,n) in Eq. (1) and the normalization c(n) in Eq. (2) are derived under a random-split process over the feature range of a single data set; averaging per-client random splits does not preserve that process when client feature ranges differ, since the averaged root split can fall between two clients' normal ranges, routing each client's data into different branches and separating clients rather than isolating anomalies. Please specify client_process_layer precisely, and provide either a formal argument or an empirical ablation showing that the averaged tree yields path-length distributions comparable to those of a centralized iForest trained on the concatenated data with the same total sample size and tree parameters.
  2. [III.A and III.C.3] All headline performance numbers are computed on the same test set that was used to select the threshold and the operating point. Section III.A states that a new synthetic testing dataset is generated from the original data with Gaussian noise plus 10% random anomalies; Section III.C.3 states that the anomaly threshold 0.8265 was 'chosen by picking a value on the precision-recall curve where an F1-Score was the highest' on that evaluation set. In addition, point E is selected via Eq. (8) as the configuration maximizing AUC-ROC and AUC-PR under the memory bound, again using the same test data. Consequently, the reported AUC-ROC 0.9996, AUC-PR 0.9728, precision 99.17%, and recall 99.8% are optimistically biased, and the 40 repeats are used only to produce averages without standard deviations or confidence intervals. Please split the data into training/validation/test, select the threshold and point E on the validation set, and report mean and standard deviation over the 40 repeats on the test set.
  3. [III.C.4] The iForest comparison is not matched in training-data size. PFLiForest points A-D use training sizes (50, 100, 150, 200), while the corresponding iForest points A'-D' use (100, 200, 300, 400). Thus the statement that the lower AUC-PR 'stems from its federated nature' confounds federated aggregation with having only half the training data; a centralized iForest trained with 50-200 samples would likely show a similar gap. Please compare PFLiForest and iForest at equal total training sample sizes and equal per-tree parameters, or at least include an iForest trained on the same per-client partitions to isolate the effect of federation.
  4. [III.A and III.C.1] The paper is inconsistent about what 'memory usage' means. Section III.A says 'maximum memory usage while training' was recorded, but Section III.C.1 and Fig. 3 equate memory usage with 'model size,' and point E is said to use 153 kB 'of memory for training.' If only the serialized isolation forest is measured, then the abstract's claim of 'maintaining a memory usage below 160 KB during model training' is unsupported, because the training process also needs the PTB-FLA runtime, communication buffers, queues, and the MicroPython interpreter. Please clarify the measurement methodology and, if the 153 kB figure is only the model size, either report end-to-end memory usage or qualify the claim accordingly.
minor comments (5)
  1. [III.A vs III.C] Section III.A lists max_depth values {4,6,8,10,12} and num_trees values {10,25,50,75,100}, but Section III.C's parameter list omits 12 and 100; please reconcile the two lists.
  2. [Abstract] The abstract claims 'over 96% accuracy in distinguishing normal from abnormal readings,' but the experimental section reports AUC-ROC, precision, and recall rather than classification accuracy; please align the terminology with the reported metrics.
  3. [References] Reference [10] contains a duplicated phrase and URL: 'https://doi.org/10.25949/25286269.v1Li, J., ...' appears twice in the same entry and should be cleaned up.
  4. [III.B] There are several typos in the metrics section, including 'clasified' in Eq. (6), 'positives samples' after Eq. (3), and 'cal lback' in Section II; please proofread the manuscript.
  5. [III.C.4] The AUC-ROC values are omitted from Fig. 5 with the explanation that they are 'nearly identical' for both algorithms, but no numeric support is given; please report the actual AUC-ROC values for points A-D and A'-D' in a table or appendix.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PFLiForest is an experimental application/evaluation paper whose performance numbers are measured, not derived from fitted parameters or self-citation chains.

full rationale

The paper does not claim to derive a novel algorithm from first principles; it presents PFLiForest as a PTB-FLA-based specialization of the externally cited FLiForest algorithm and reports experimental measurements of AUC-ROC, AUC-PR, memory, and time. The anomaly score formulas (Eqs. 1–2) are the standard Isolation Forest equations from Liu et al. (2008), an independent external source, and no paper equation is defined in terms of another result such that a claim reduces to its own input. The threshold value 0.8265 is chosen on the test set's precision-recall curve, which is an evaluation-protocol issue that can inflate reported precision/recall, but it is explicitly disclosed and the headline AUC-ROC and AUC-PR metrics are threshold-independent. The self-citations to PTB-FLA, MPT-FLA, CSP verification, and the development paradigm support the implementation framework but are not load-bearing for the central performance claim, which rests on 40-run experiments and a comparison against a standard iForest baseline. The unspecified client_process_layer is a missing implementation detail and a threat to reproducibility, but not a circular reduction. Even the selection of point E via Eq. (8) is a model-selection procedure on measured memory and AUC values, not a fitted quantity being renamed as a prediction. Under the stated rules requiring a quotable reduction, the paper contains no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new theoretical entities. The free parameters are standard hyperparameters plus a threshold fitted to the test set. The key axioms are the validity of the standard isolation forest formulas, the soundness of averaging client splits, and the representativeness of the synthetic test data.

free parameters (4)
  • num_trees = 10, 25, 50, 75, 100
    Number of trees in the isolation forest, chosen by hand as a hyperparameter.
  • max_depth = 4, 6, 8, 10, 12
    Maximum tree depth, chosen by hand.
  • training_data_size = 50, 100, 150, 200
    Amount of training data per configuration, chosen by hand.
  • anomaly_threshold = 0.8265
    Threshold selected on the synthetic test set to maximize F1 score, so it is fitted to the evaluation data.
assumptions (4)
  • standard math The standard isolation forest anomaly score (Eq. 1-2) is valid for anomaly detection.
    Adopted from Liu et al. [7] without modification.
  • ad hoc to paper Averaging client-computed split values yields a globally valid isolation tree.
    Introduced in Algorithm 4 as server_aggregate_layer; no theoretical justification or comparison to other aggregation methods is given.
  • domain assumption The synthetic test set, generated by adding Gaussian noise and random anomalies to the original temperature data, provides a valid evaluation of anomaly detection.
    Section III.A describes the generation; the representativeness of this test set for real-world anomalies is assumed.
  • domain assumption PTB-FLA's fl_centralized correctly implements the federated communication protocol.
    The paper relies on the correctness of their own framework, citing CSP verification [5] for the generic algorithm but not for this specialization.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Federated Isolation Forest for Efficient Anomaly Detection on Edge IoT Systems." pith.science (2026). https://pith.science/paper/T6JR6X5K

@misc{pith2026250605138,
  author       = {Pith},
  title        = {Pith review of: Federated Isolation Forest for Efficient Anomaly Detection on Edge IoT Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T6JR6X5K}},
  note         = {Machine review of arXiv:2506.05138}
}
read the original abstract

Recently, federated learning frameworks such as Python TestBed for Federated Learning Algorithms and MicroPython TestBed for Federated Learning Algorithms have emerged to tackle user privacy concerns and efficiency in embedded systems. Even more recently, an efficient federated anomaly detection algorithm, FLiForest, based on Isolation Forests has been developed, offering a low-resource, unsupervised method well-suited for edge deployment and continuous learning. In this paper, we present an application of Isolation Forest-based temperature anomaly detection, developed using the previously mentioned federated learning frameworks, aimed at small edge devices and IoT systems running MicroPython. The system has been experimentally evaluated, achieving over 96% accuracy in distinguishing normal from abnormal readings and above 78% precision in detecting anomalies across all tested configurations, while maintaining a memory usage below 160 KB during model training. These results highlight its suitability for resource-constrained environments and edge systems, while upholding federated learning principles of data privacy and collaborative learning.

Figures

Figures reproduced from arXiv: 2506.05138 by the authors.

Figure 1
Figure 1. Federated anomaly detection system architecture The nodes communicate via PTB-FLA testbed instances running on each of them. The function fl_centralized implements a generic centralized federated learning algorithm using callback functions, following the Single Program Multiple Data (SPMD) pattern. B. System Behaviour Due to the layer-by-layer nature of the way the model training is conducted in the algorithm FLiFor… view at source ↗
Figure 2
Figure 2. The UML class diagram for the class IsolationTreeNode C. System inference The isolation forest inference is done by computing an anomaly score for a given data point by traversing the isolation trees and averaging path lengths across an isolation forest. The anomaly score is computed as: 𝑠(𝑥, 𝑛) = 2 − 𝐸(ℎ(𝑥)) 𝑐(𝑛) () 𝑐(𝑛) = 2(𝑙𝑛(𝑛 −1) +0.577)− 2(𝑛−1) 𝑛 () Where E(h(x)) is the average path length across the isolati… view at source ↗
Figure 3
Figure 3. shows AUC-ROC and AUC-PR against the memory usage values for the 4 points A, B, C, and D. The goal is to fit the model into the RPi Pico device. Therefore, we search for point E with the maximum values of AUC-ROC and AUC-PR where memory usage is ~160 kB: 𝐸 = 𝑀(max(𝐴𝑈𝐶𝑅𝑂𝐶) , max(𝐴𝑈𝐶𝑃𝑅)) < 160 () [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: AUC-PR of PFLiForest and iForest versus the memory usage Calculations are given for the points A, B, C, and D for PFLiForest and the corresponding points A’ (4, 10, 100), B’ (6, 25, 200), C’ (8, 50, 300), and D’ (10, 75, 400) for iForest. The point E and the correspond…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. EcoDefender: Energy-Efficient Hybrid Anomaly Detection for IoT Edge Gateways

    cs.CR 2025-11 reject novelty 4.0 of 10

    A hybrid autoencoder-isolation-forest IoT edge detector reports 94% accuracy with low overhead and carbon accounting, but its key formulas and energy/carbon numbers are internally inconsistent.

Reference graph

Works this paper leans on

11 extracted references · 6 canonical work pages · cited by 1 Pith paper

  1. [1]

    https://www.project -tardis.eu/, 2023 (accessed 1 February 2024)

    TaRDIS: Trustworthy and Resilient Decentralised Intelligence For Edge Systems. https://www.project -tardis.eu/, 2023 (accessed 1 February 2024)

    work page 2023

  2. [2]

    Communication-Efficient Learning of Deep Networks from Decentralized Data,

    H.B. McMahan, E. Moore, D. Ramage, S. Hampson, B. A. y Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” in Proc. of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), JMLR: W&CP volume 54, pp. 1-10, 2017

    work page 2017

  3. [3]

    Popovic, M

    M. Popovic, M. Popovic, I. Kastelan, M. Djukic, and S. Ghilezan, A Simple Python Testbed for Federated Learning Algorithms, 2023, in: Proceedings of the 2023 Zooming Innovation in Consumer Technologies Conference, 2023, pp. 148 -153, https://doi.org/10.1109/ZINC58345.2023.10173859

    arXiv 2023

  4. [4]

    MicroPython Testbed for Federated Learning Algorithms,

    M. Popovic, M. Popovic, I. Kastelan, M. Djukic, I. Basicevic and P. Vasiljevic, "MicroPython Testbed for Federated Learning Algorithms," 2024 32nd Telecommunications Forum (TELFOR), Belgrade, Serbia, 2024, pp. 1 -4, doi: 10.1109/TELFOR63250.2024.10819071

    arXiv 2024

  5. [5]

    Prokić, S

    I. Prokić, S. Ghilezan, S. Kašterović, M. Popovic, M. Popovic, I. Kaštelan, Correct orchestration of Federated Learning generic algorithms: formalisation and verification in CSP, in: J. Kofron, T. Margaria, C. Seceleanu (Eds.), Engineering of ComputerBased Systems, Lecture Notes in Computer Science, Vol. 14390, Springer, Cham, 2024, pp. 274 –288, https://...

    work page doi:10.1007/978-3-031-49252- 2024

  6. [6]

    Popovic, M

    M. Popovic, M. Popovic, I. Kastelan, M. Djukic, and I. Basicevic, A Federated Learning Algorithms Development Paradigm, in: J. Kofron, T. Margaria, C. Seceleanu (Eds.), Engineering of Computer -Based Systems, Lecture Notes in Computer Science, Vol. 14390, S pringer, Cham, 2024, pp. 26 –41, https://doi.org/10.1007/978 -3-031- 49252- 5_4

    doi:10.1007/978 2024

  7. [7]

    Isolation Forest,

    F. T. Liu, K. M. Ting and Z. -H. Zhou, "Isolation Forest," 2008 Eighth IEEE International Conference on Data Mining, Pisa, Italy, 2008, pp. 413-422, doi: 10.1109/ICDM.2008.17

    doi:10.1109/icdm.2008.17 2008

  8. [8]

    Leveraging an Isolation Forest to Anomaly Detection and Data Clustering,

    Yepmo, V., Smits, G., Lesot, M. -J., & Pivert, O., "Leveraging an Isolation Forest to Anomaly Detection and Data Clustering," Data & Knowledge Engineering, vol. 151, no. C, pp. 102302, May 2024. [Online]. Available: https://doi.org/10.1016/j.datak.2024.102302

    arXiv 2024

Show all 11 references
  1. [9]

    Layered isolation forest: A multi-level subspace algorithm for improving isolation forest,

    Liu, T., Zhou, Z., & Yang, L., "Layered isolation forest: A multi-level subspace algorithm for improving isolation forest," Neurocomputing, vol. 581, no. C, pp. 127525, May 2024. [Online]. Available: https://doi.org/10.1016/j.neucom.2024.127525

    2024

  2. [10]

    Federated anomaly detection with Isolation Forest in the IoT network,

    Li, J., "Federated anomaly detection with Isolation Forest in the IoT network," Ph.D. thesis, Macquarie Univ., 2024. [Online]. Available: https://doi.org/10.25949/25286269.v1Li, J., "Federated anomaly detection with Isolation Forest in the IoT network," Ph .D. thesis, Macquari...

    2024 doi

  3. [11]

    Xiang, H., Zhang, X., Xu, X., Beheshti, A., Qi, L., Hong, Y., & Dou, W. (2024). Federated learning-based anomaly detection with isolation forest in the IoT -edge continuum. ACM Transactions on Multimedia Computing, Communications, and Applications, Just Acc epted. https://doi....

    2024 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.