Pith. sign in

REVIEW 4 major objections 6 minor 19 references

BotDetect: A Decentralized Federated Learning Framework for Detecting Financial Bots on the EVM Blockchains

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read BotDetect reports that a federated learning framework coordinated by smart contracts detects financial bots on EVM blockchains with a binary F1 of 0.99 and multiclass F1 of 0.97, outperforming centralized baseline models while keeping…

desk verdict Plausible decentralized FL architecture and real on-chain benchmark, but the reported detection accuracy is unsupported because SMOTE is applied before the train/test split. read the letter →

arxiv 2501.12112 v1 pith:MPVR2YRN submitted 2025-01-21 cs.CR cs.DC

Ahmed Mounsf Rafik Bendada , Abdelaziz Amara Korba , Mouhamed Amine Bouchiha , Yacine Ghamri-Doudane This is my paper · ORCID
classification cs.CRcs.DC
keywords financialbotdetectionfederatedlearningdecentralizedEVMblockchainssmartcontractaggregationMEVbotsSMOTEtransactionanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that bot detection on EVM blockchains can be run as a decentralized federated learning process, in which nodes train locally on their own transaction histories and a smart contract aggregates the model updates. The authors report that this design detects financial bots with a binary F1-score of 0.99 and a multiclass F1-score of 0.97, beating centralized machine learning baselines from the dataset they build on. If this result holds, it would give DeFi networks a privacy-preserving way to identify arbitrage, sandwich, and liquidation bots without collecting user data into a central repository. The reader should care because bot exploitation currently costs users over a billion dollars through front-running and similar attacks, and existing rule-based tools cannot adapt to new bot strategies.

What carries the argument

The central mechanism is on-chain federated averaging. Each training client pulls the current global model from the blockchain, trains a fully connected neural network locally for several epochs using SGD, then submits its weight vector to the AG smart contract; once K updates arrive, the contract computes the average (optionally weighted by each client's dataset size) and publishes the new global model for the next round. This replaces a central parameter server with a transparent, auditable smart contract, and the paper's Algorithm 1 specifies the aggregation logic. The same mechanism, together with a permissioned QBFT consensus layer, is what the paper argues makes bot detection both decentralized and private.

What would settle it

Re-run the binary and multiclass evaluations on the original 270 labeled addresses, or on newly labeled bot addresses, without SMOTE expansion; if F1 drops materially below 0.99/0.97, the headline accuracy is an artifact of the synthetic test set. A second observation: submit a deliberately poisoned weight update from one client and check whether the on-chain average remains clean.

Watch

Extended reading notes

Core claim

The paper's central claim is that a decentralized federated learning framework, orchestrated by a smart contract, can detect financial bots on EVM blockchains more accurately than centralized training while preserving data privacy. On the Niedermayer et al. dataset of 270 labeled Ethereum accounts, the framework reports 0.99 accuracy and F1 in binary bot/human classification and 0.96 accuracy with 0.97 F1 in multiclass classification across arbitrage, liquidation, sandwich, and non-MEV categories. The authors attribute the improvement over centralized models to data diversity: each client trains on a different private subset, and the aggregated global model generalizes better than a model trained on any single copy. They also report on-chain performance with a QBFT permissioned network, reaching a maximum throughput of 255 transactions per second for the update submission function.

Load-bearing premise

The result stands or falls on treating SMOTE-generated synthetic examples as genuine bot transactions, since the reported test accuracy is measured on data that was synthesized, not on newly observed real bots.

Editorial extensions

If this is right

  • A working BotDetect would let DeFi nodes cooperate on bot detection without ever sharing raw transaction histories, since only model weights cross the network.
  • The reported accuracy suggests that the behavioral features in the underlying dataset carry enough signal for a small neural network, so the approach could be extended to other EVM chains like BNB Smart Chain and Solana with similar feature pipelines.
  • Since aggregation happens on-chain, the global model's provenance is public and tamper-evident, which fits the transparency requirements of permissioned consortia.
  • The measured throughput of roughly 255 TPS for submitting updates indicates the on-chain overhead is not prohibitive for periodic retraining, provided model sizes stay small.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 0.99/0.97 accuracy figures are likely optimistic because the test set was built after SMOTE expansion, so synthetic test points share neighborhoods with synthetic training points; a holdout evaluation on the original 270 labeled addresses would give a more realistic number.
  • A natural next stress test would be to inject Byzantine or malicious updates; the presented aggregation contract averages every submitted update equally, so it has no defense against a poisoned client.
  • The framework's assumption that all participating nodes are reliable and cooperative excludes the most adversarial scenario in public blockchains, so the design as presented fits permissioned settings better than open ones.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes BotDetect, a decentralized federated learning (DFL) framework for detecting financial bots on EVM blockchains. In the proposed design, clients train a fully connected neural network on local transaction-derived features, and model updates are aggregated on-chain by a smart contract using a permissioned QBFT consensus. The authors evaluate detection accuracy using the 270-address dataset of Niedermayer et al., augmented via SMOTE, and report binary F1 of 0.99 and multiclass F1 of 0.97 with 3-5 clients. They also benchmark the smart-contract functions submitUpdate and aggregateModel with Hyperledger Caliper, reporting a maximum throughput of about 255 TPS for submitUpdate at 1s block time.

Significance. The problem is timely and relevant: financial bots on EVM blockchains cause measurable harm, and a privacy-preserving decentralized detection framework is a worthwhile goal. The smart-contract orchestration of federated learning and the on-chain aggregation design are architecturally interesting, and the throughput/latency benchmark in Section IV-C is a useful empirical contribution. However, the paper's headline claim of near-99% detection accuracy is not supported by the current experimental protocol, because the SMOTE-based augmentation is applied before the train/test split, contaminating the test set with synthetic samples generated from training neighborhoods. As a result, the reported F1 scores are not credible estimates of performance on unseen real addresses. The comparison against centralized baselines is also not apples-to-apples. The core contribution therefore rests on an invalid evaluation, and the paper needs a substantial revision of its experimental methodology.

major comments (4)
  1. [Section IV-A, Tables II-IV] The dataset is expanded with SMOTE before the 70/30 split. Because each SMOTE sample is generated by interpolating among its k=5 nearest neighbors in the original feature space, the synthetic test samples lie close to training samples and do not provide an independent test of generalization. For the binary case, the expanded dataset contains 10,000 synthetic examples plus 270 real addresses, so a 30% test slice is roughly 97% synthetic. The F1 scores of 0.99 (Table III) and 0.97 (Table IV) are therefore measured on a non-independent test set and do not support the claim that BotDetect detects previously unseen real bot addresses. The split must be performed on the original 270 addresses first, SMOTE applied only to the training partition, and metrics reported on the untouched real test fold, ideally with multiple random seeds and confidence intervals.
  2. [Tables III and IV] The comparison in Tables III and IV is not apples-to-apples: the Random Forest, Gradient Boosting, and AdaBoost numbers are taken from [1], where they were obtained on the original 270-address dataset without SMOTE augmentation, while 'Our Work' is trained on SMOTE-expanded data and evaluated on a synthetic-heavy test set. This conflates the detection method with the data-augmentation protocol. To support the claimed superiority, the authors must train all baselines on the same augmented training partition and evaluate them on the same untouched real test partition.
  3. [Table II] The centralized baseline in Table II is insufficiently specified. The text says 'identical setups with 4 and 5 clients' but does not state whether the centralized model was trained on the entire training portion or on only a single client's subset. Since DFL aggregates updates from all clients, a centralized model trained on all data would ordinarily be expected to perform at least comparably; the large gap (0.877 vs 0.984) suggests the centralized model may have been trained on a smaller subset or with different hyperparameters. The authors should specify the exact training data and hyperparameters for the centralized baseline, and if it was intentionally limited, explain why the comparison is fair.
  4. [Abstract and Section V] The abstract and conclusion claim that the framework maintains 'robustness', but no experiment addresses robustness: there are no tests with non-IID client data, client dropout, poisoning attacks, or varying client participation. Either add such experiments or remove the robustness claim from the abstract and conclusion.
minor comments (6)
  1. [Section III-B, Eq. (5)-(6), Algorithm 1] Equation (5) presents unweighted averaging and Eq. (6) presents data-size-weighted aggregation, but Algorithm 1 implements simple averaging. Please clarify which aggregation rule is actually used and under what conditions the weighted variant applies.
  2. [Section II] Reference [5], cited for graph-theoretic metrics for classifying Ethereum wallets, is a semi-supervised learning survey by Xiaojin Zhu; it does not appear to support the stated claim. Please verify the intended citation.
  3. [Figure 4] The two-panel figure with both throughput and latency curves is hard to read, especially the latency y-axis labels. Consider using four separate subfigures or larger fonts.
  4. [Section IV-A] The multiclass labels 'Arbitrage, Liquidation, Sandwich, and Non-MEV' are introduced only in the preprocessing section; please introduce them before the multiclass results in Table IV, and define what 'Non-MEV' includes.
  5. [General] No link to code or smart-contract artifacts is provided. Making these available would substantially improve reproducibility.
  6. [Section V] There is a minor typo in the conclusion: 'transaction' history should be 'transaction history'.

Circularity Check

1 steps flagged · score 6.0 of 10

The central accuracy claim is evaluated on a SMOTE-generated test set constructed from the training data, making the headline F1 scores partly self-referential; no load-bearing self-citation circularity otherwise.

  1. other [Section IV-A, Data Preprocessing; Tables II-IV]
    "Given the limited number of samples in the dataset and our need for additional examples to support federated training, we expand the dataset using the Synthetic Minority Over-sampling Technique (SMOTE) [18] with κ=5. We generated 5000 samples for each data class ... The dataset is then split into 70% training and 30% testing."

    The 30% test slice is created after SMOTE expansion, so it is dominated by synthetic points that are interpolations among κ=5 nearest neighbors of the original (training) samples. Those synthetic points lie by construction in the same local neighborhoods as the training data, so the model is not being evaluated on independent, unseen real addresses. The reported binary F1 of 0.99 (Table III) and multiclass F1 of 0.97 (Table IV) are therefore measures of how well the model reproduces the SMOTE manifold the authors themselves generated, not predictions of detection on new real bot transactions. The test performance is forced by the construction of the test set rather than by generalization.

full rationale

The paper does not derive a theoretical prediction from first principles; it reports an empirical evaluation of a DFL framework. The only 'prediction' is classification accuracy on a held-out test set, and that held-out property is undermined by the SMOTE-before-split protocol. Because SMOTE creates each synthetic sample by interpolating among κ=5 nearest neighbors, a test sample drawn from the expanded set is by construction close to training samples, so the high F1 is a partial artifact of the augmentation process. This is the one significant circular/self-referential step. The self-citations ([9], [10], [19]) are background or future-work references and are not load-bearing: no uniqueness theorem is imported, and no ansatz is justified solely by the authors' prior work. The on-chain throughput/latency evaluation (Section IV-C) is an independent, self-contained benchmark and is not affected by the SMOTE issue. The comparison to Random Forest, Gradient Boosting, and AdaBoost is additionally weakened because the baselines were trained on the original dataset rather than the SMOTE-expanded data, but that is a fairness issue rather than a circularity. Overall, the central accuracy claim is partially circular in its evaluation construction, while the framework's blockchain performance claims remain independent.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central accuracy claim rests on SMOTE-expanded synthetic data (Section IV-A), which is the largest hidden cost. The framework also assumes cooperative, reliable nodes and a permissioned dedicated chain (Section III-A). No new physical or conceptual entities are introduced; the smart contract aggregator is a standard software component.

free parameters (5)
  • SMOTE oversampling size = 5000 per class
    Chosen by hand to expand the 270-address dataset; directly determines training and test composition, and inflates apparent accuracy when split after oversampling.
  • SMOTE neighbors k = 5
    Neighborhood size for generating synthetic samples; a convenience choice that shapes the synthetic distribution.
  • Learning rate = 0.00001
    Adam optimizer learning rate; chosen without tuning and possibly responsible for underfitting of the centralized baseline.
  • Hidden layer size constant l = unspecified (1-10)
    The hidden node count n = sqrt(m+k)+l depends on l, but the paper never states its value; the model architecture is therefore under-specified.
  • Number of FL clients = 3, 4, 5
    Tested configurations; performance varies slightly across them but no statistical analysis is given.
assumptions (4)
  • domain assumption Participating nodes in the federated learning network are reliable and cooperative.
    Stated in Section III (system model). Rule-based on-chain aggregation has no defense against malicious weight submissions, so the security claim inherits this assumption.
  • domain assumption The feature set and labels from Niedermayer et al. [1] are sufficient to capture bot behavior.
    Section III-B reuses the prior dataset and features without re-validation; the neural network's input quality depends on these features.
  • ad hoc to paper SMOTE-generated samples are representative of real bot and human behavior.
    The whole evaluation relies on 5000 synthetic samples per class; this is introduced ad hoc and is not supported by any external validation.
  • domain assumption The dedicated blockchain is isolated from the primary network and QBFT provides integrity.
    Section III-A assumes a permissioned setup on a separate chain; the detection accuracy result is independent of this, but the decentralization claim depends on it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BotDetect: A Decentralized Federated Learning Framework for Detecting Financial Bots on the EVM Blockchains." pith.science (2026). https://pith.science/paper/MPVR2YRN

@misc{pith2026250112112,
  author       = {Pith},
  title        = {Pith review of: BotDetect: A Decentralized Federated Learning Framework for Detecting Financial Bots on the EVM Blockchains},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MPVR2YRN}},
  note         = {Machine review of arXiv:2501.12112}
}
read the original abstract

The rapid growth of decentralized finance (DeFi) has led to the widespread use of automated agents, or bots, within blockchain ecosystems like Ethereum, Binance Smart Chain, and Solana. While these bots enhance market efficiency and liquidity, they also raise concerns due to exploitative behaviors that threaten network integrity and user trust. This paper presents a decentralized federated learning (DFL) approach for detecting financial bots within Ethereum Virtual Machine (EVM)-based blockchains. The proposed framework leverages federated learning, orchestrated through smart contracts, to detect malicious bot behavior while preserving data privacy and aligning with the decentralized nature of blockchain networks. Addressing the limitations of both centralized and rule-based approaches, our system enables each participating node to train local models on transaction history and smart contract interaction data, followed by on-chain aggregation of model updates through a permissioned consensus mechanism. This design allows the model to capture complex and evolving bot behaviors without requiring direct data sharing between nodes. Experimental results demonstrate that our DFL framework achieves high detection accuracy while maintaining scalability and robustness, providing an effective solution for bot detection across distributed blockchain networks.

Figures

Figures reproduced from arXiv: 2501.12112 by the authors.

Figure 1
Figure 1. Workflow of the proposed framework: Step-1. get initial global model; Step-2. local training; Step-3. submit update; Step-4. aggregation; Step-5. write global model and repeat. III. PROPOSED SOLUTION We introduce an advanced bot detection framework employing deep learning (DL) models trained on transaction histories captured independently at each blockchain node. Leveraging federated learning, this approach ensures … view at source ↗
Figure 2
Figure 2. Heatmap of Accuracy and Loss Over Rounds for the [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Confusion Matrices: DFL (4 and 5 clients) vs. centralized [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Latency and Throughput comparison under two workload [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 17 canonical work pages

  1. [1]

    Detecting financial bots on the ethereum blockchain

    Thomas Niedermayer, Pietro Saggese, and Bernhard Haslhofer. Detecting financial bots on the ethereum blockchain. In Companion Proceedings of the ACM on Web Conference, pages 1742–1751, 2024

    work page 2024

  2. [2]

    Miners as inter- mediaries: extractable value and market manipulation in crypto and defi

    Raphael Auer, Jon Frost, and Jose Maria Vidal Pastor. Miners as inter- mediaries: extractable value and market manipulation in crypto and defi. Technical report, Bank for International Settlements, 2022

    work page 2022

  3. [3]

    Ready, aim, snipe! analysis of sniper bots and their impact on the defi ecosystem

    Federico Cernera, Massimo La Morgia, Alessandro Mei, Alberto Maria Mongardini, and Francesco Sassi. Ready, aim, snipe! analysis of sniper bots and their impact on the defi ecosystem. In Companion Proceedings of the ACM Web Conference 2023, pages 1093–1102. ACM, 2023

    work page 2023

  4. [4]

    Defining user spectra to classify ethereum users based on their behavior

    Gianluca Bonifazi, Enrico Corradini, Domenico Ursino, and Luca Virgili. Defining user spectra to classify ethereum users based on their behavior. Journal of Big Data, 9(1):37, 2022

    work page 2022

  5. [5]

    Semi-supervised learning literature survey

    Xiaojin (Jerry) Zhu. Semi-supervised learning literature survey. Technical report, University of Wisconsin-Madison Department of Computer Sci- ences, 2005

    work page 2005

  6. [6]

    Token Spammers, Rug Pulls, and Sniper Bots: An Analysis of the Ecosystem of Tokens in Ethereum and in the Binance Smart Chain (BNB)

    Federico Cernera, Massimo La Morgia, Alessandro Mei, and Francesco Sassi. Token Spammers, Rug Pulls, and Sniper Bots: An Analysis of the Ecosystem of Tokens in Ethereum and in the Binance Smart Chain (BNB). In 32nd USENIX Security Symposium (USENIX Security 23), pages 3349–3366, 2023

    work page 2023

  7. [7]

    The technology of decentralized finance (defi)

    Raphael Auer, Bernhard Haslhofer, Stefan Kitzler, Pietro Saggese, and Friedhelm Victor. The technology of decentralized finance (defi). Digital Finance, 3(2):425–453, 2023

    work page 2023

  8. [8]

    A model for detecting cryptocurrency transactions with discernible purpose

    Hyochang Baek, Junhyoung Oh, Chang Yeon Kim, and Kyungho Lee. A model for detecting cryptocurrency transactions with discernible purpose. In 2019 Eleventh International Conference on Ubiquitous and Future Networks (ICUFN), pages 713–717. IEEE, 2019

    work page 2019

Show all 19 references
  1. [9]

    Federated learning for zero-day attack detection in 5g and beyond v2x networks

    Abdelaziz Amara Korba, Abdelwahab Boualouache, Bouziane Brik, Rabah Rahal, Yacine Ghamri-Doudane, and Sidi Mohammed Senouci. Federated learning for zero-day attack detection in 5g and beyond v2x networks. In ICC 2023 - IEEE International Conference on Communications, pages 113...

    2023

  2. [10]

    Zero-x: A blockchain-enabled open-set federated learning frame- work for zero-day attack detection in iov

    Abdelaziz Amara Korba, Abdelwahab Boualouache, and Yacine Ghamri- Doudane. Zero-x: A blockchain-enabled open-set federated learning frame- work for zero-day attack detection in iov. IEEE Transactions on Vehicular Technology, 73(9):12399–12414, 2024

    2024

  3. [11]

    Rajasekaran

    Bibhudatta Samal and S. Rajasekaran. Federated learning-based intrusion detection in distributed environments. In the International Conference on Distributed Computing, pages 123–130, 2021

    2021

  4. [12]

    Federated intrusion detection in smart iot networks

    Xiaofeng Zhou and Le Wang. Federated intrusion detection in smart iot networks. IEEE Internet of Things Journal, 8(5):2345–2358, 2022

    2022

  5. [13]

    Decentralized federated learning: A survey and perspective

    Liangqi Yuan, Ziran Wang, Lichao Sun, S Yu Philip, and Christopher G Brinton. Decentralized federated learning: A survey and perspective. IEEE Internet of Things Journal, 2024

    2024

  6. [14]

    Blockchain-based federated learning for secure and scalable learning

    Xiaojie Huang and Yimin Yu. Blockchain-based federated learning for secure and scalable learning. Journal of Information Security and Appli- cations, 55:102636, 2020

    2020

  7. [15]

    Blockchain-empowered federated learning: Challenges, solutions, and fu- ture directions

    Juncen Zhu, Jiannong Cao, Divya Saxena, Shan Jiang, and Houda Ferradi. Blockchain-empowered federated learning: Challenges, solutions, and fu- ture directions. ACM Computing Surveys, 55(11):1–31, 2023

    2023

  8. [16]

    Blockchain-based federated learning framework for decentralized applications

    Lei Xu and Wei Zhang. Blockchain-based federated learning framework for decentralized applications. IEEE Transactions on Blockchain, 3:143-153, 2021

    2021

  9. [17]

    HotStuff: BFT consensus in the lens of blockchain

    Maofan Yin, Dahlia Malkhi, Michael K Reiter, Guy Golan Gueta, and Ittai Abraham. HotStuff: BFT consensus in the lens of blockchain. arXiv preprint arXiv:1803.05069, 2018

    2018 arXiv

  10. [18]

    Smote: Synthetic minority over-sampling technique

    Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16:321–357, 2002

    2002

  11. [19]

    RollupTheCrowd: Leveraging ZkRollups for a Scalable and Privacy-Preserving Reputation-Based Crowd- sourcing Platform

    Ahmed Mounsf Rafik Bendada, Mouhamed Amine Bouchiha, Mourad Rabah, and Yacine Ghamri-Doudane. RollupTheCrowd: Leveraging ZkRollups for a Scalable and Privacy-Preserving Reputation-Based Crowd- sourcing Platform. In IEEE 48th Annual Computers, Software, and Appli- cations Confe...

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.