Pith. sign in

REVIEW 3 major objections 4 minor 17 references

FBID: Adaptive Personalized Federated Learning for Robust Out-of-Distribution Attack Detection in IoT Networks

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FBID claims a server-side bandit plus trust-based blending of global and local models lifts per-client out-of-distribution attack detection by up to 7.66% and F1 by 5.08% over stable baselines while preserving in-distribution accuracy.

desk verdict A sensible server-side personalization controller that is undermined by an OOD evaluation whose test set overlaps the training classes, leaving the headline robustness claim unproven. read the letter →

arxiv 2608.04073 v1 pith:FLIATBUE submitted 2026-08-04 cs.CR cs.AIcs.LG

An Khanh Bui , Cong Thanh Nguyen , Hoang-Anh Pham , Hoang Thai Dinh , Diep N. Nguyen This is my paper · ORCID
classification cs.CRcs.AIcs.LG
keywords FederatedLearningPersonalizedIntrusionDetectionIoTSecurityOut-of-DistributionContextualMulti-ArmedBanditNon-IIDDataModelRobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to fix a specific failure of personalized federated learning for intrusion detection: when each IoT client tunes its own model, skewed local data can over-personalize the model and blind it to attacks it has not seen. FBID moves the personalization dial to the server. A contextual bandit picks a per-client training budget, and a trust score derived from validation reward controls how much each client's final model leans on the global model. On CICIoT2023 with 10 heterogeneous clients and 21 unseen attack classes, the paper reports up to 7.66% higher per-client OOD detection rate and 5.08% higher F1 than the best stable baseline. If the result holds, it would mean server-side oversight is a cheap way to keep global attack knowledge without giving up local adaptation.

What carries the argument

The central machinery is the server-side controller. A LinUCB contextual multi-armed bandit selects each client's local epoch budget from {1, 2, 3, 5}; the reward combines the improvement in AUC on a server-held validation set from a shadow update, the client's local F1, and a drift penalty on the scaled weight update. An exponential moving average of the sigmoid of the z-scored reward gives a trust score tau_k, and alpha_k = 1 - tau_k interpolates between global and local models: wbar_k = alpha_k w_G + (1 - alpha_k) w_k. This decouples the optimization depth from the degree of model deviation, so a single reward signal does not have to solve the credit-assignment problem of attributing poor performance to either insufficient training or excessive personalization.

What would settle it

Check the class lists: if DoS-TCP Flood, DoS-SYN Flood, DDoS-UDP Flood, DDoS-ICMP Flood, DNS Spoofing, or DictionaryBruteForce appear in any client's training profile, then the stress set is not out-of-distribution. Re-running the comparison with a stress set built from attack families excluded from all training data, and reporting the validation set's class composition, would settle whether the claimed robustness to unseen attacks is real.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that server-side supervision of personalization fixes the out-of-distribution failure mode of client-driven personalized federated learning. The authors show that with a single per-client reward, a contextual bandit controlling only the local epoch budget and a trust-derived blending coefficient can keep personalized models close enough to the global model on low-quality or benign-skewed clients while allowing high-quality clients to specialize. In their experiments, reliable clients converge to alpha around 0.27 and skewed clients to alpha around 0.73, whereas the client-driven CBC baseline over-personalizes benign-heavy clients (missing nearly all attacks) and over-relies on the global model for information-rich gateways. FBID reports aggregated OOD F1 of 0.680 and detection rate of 0.518, versus 0.660 and 0.494 for the strongest stable baseline FedALA, with per-client relative gains up to 5.08% in F1 and 7.66% in detection rate.

Load-bearing premise

The central claim collapses if the OOD stress set is not actually out-of-distribution: Table 1's 'unseen' classes overlap with the client training profiles described in Section IV.A.1 (volumetric DoS/DDoS for gateways, DNS/ARP spoofing and brute force for hubs), and the paper does not establish that the server validation set is class-disjoint from the test set.

Editorial extensions

If this is right

  • Deploying FBID would let a federation keep the in-distribution accuracy of personalized learning while avoiding the near-complete attack-detection collapse that client-side methods show on benign-heavy clients.
  • The overhead is small enough for practice: one scalar epoch budget and one blending coefficient per client per round, plus a modest server-side validation set, with no change to the client model family.
  • The trust trajectories give operators a per-client view of update quality: reliable clients end with alpha around 0.27, while skewed clients are held near alpha 0.73, keeping them anchored to the global model.
  • Because the controller consumes only scalar performance signals, the same supervision mechanism could in principle be applied to sequence-based or deeper intrusion detection architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The headline margin is likely sensitive to the exact composition of the stress set; a clean test would construct OOD classes by removing entire attack families from all training partitions and then measure the FBID-versus-FedALA gap.
  • The same server-side bandit plus trust-blending design could transfer to other federated domains with a public validation corpus, such as fraud or malware detection, where OOD robustness matters as much as local accuracy.
  • Because the server-side validation set is the only channel through which the server observes client behavior, the framework's behavior under different validation-set distributions is an open question; a sensitivity study over validation sources would sharpen the practical guidance.
  • The trust score's neutral prior of 0.5 means early rounds treat all clients equally, so the bandit's exploration phase is load-bearing for convergence speed and could be tuned per deployment.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes FBID, a personalized federated learning framework in which a server-side contextual bandit adaptively assigns per-client local training budgets and a trust-based mechanism blends global and local models into client-specific personalized models. The stated goal is to prevent over-personalization and preserve out-of-distribution (OOD) attack-detection performance in heterogeneous IoT intrusion detection. The paper evaluates FBID on the CICIoT2023 dataset with 10 non-IID clients and reports improvements in OOD Detection Rate and F1 over baselines such as FedALA, APFL, Ditto, and CBC. The core idea of server-side control over personalization is interesting, but the experimental evaluation contains a load-bearing flaw: the OOD stress test is not actually out-of-distribution relative to the training profiles described in the same section, and the server-side validation set used to drive both the bandit and the trust score is unspecified. These issues directly undermine the paper's central claims.

Significance. If the claims were supported, FBID would provide a practical and low-overhead mechanism for mitigating the OOD degradation that can result from client-driven personalization in federated intrusion detection. The framework's separation of optimization (epoch budget via bandit) from interpolation (blending coefficient via trust) is a reasonable design response to the credit-assignment problem, and the communication overhead is genuinely small. However, the evidence presented does not establish the headline claims because the OOD stress set overlaps with training classes and the reward used for both control loops is computed on an unspecified validation set. The method is interesting enough to warrant a major revision, but the evaluation must be redone with a truly disjoint held-out attack-class set and a fully specified validation set.

major comments (3)
  1. [IV.A.1, Table 1] The OOD stress test is not out-of-distribution with respect to the training data as described. Section IV.A.1 states that 'this OOD set introduces novel attack types that are entirely absent from the clients' training data,' but the same section assigns Nodes 1-3 training data 'focused on volumetric DoS/DDoS (TCP, SYN, UDP, ICMP),' Nodes 4-6 data featuring 'DNS/ARP Spoofing, Brute Force,' and Nodes 9-10 a mix of DoS/DDoS and slow-rate attacks. Table 1's stress set contains DoS-TCP Flood, DoS-SYN Flood, DDoS-UDP Flood, DDoS-ICMP Flood, DDoS-SlowLoris, MITM-ArpSpoofing, DNS Spoofing, and DictionaryBruteForce. The majority of the stress-set attack classes are therefore present in at least some clients' training distributions. The paper provides no class-level or client-level overlap analysis, so the reported gains in Tables 2 and 3 cannot be attributed to robustness to previously unseen attack classes; they may reflect in-distribution or covariate-shift performance. This directly contradicts the abstract and conclusion claims about robustness to previously unseen attacks.
  2. [II.A, III.A, III.B (Eq. (3)-(6), Algorithm 1)] The server-side validation set D_val is unspecified and is load-bearing for both control signals. D_val is introduced in Section II.A, but its size, class composition, and provenance are never given. The reward r_k in Eq. (3) depends on delta_AUC computed on D_val (Eq. (4)), and this reward updates both the LinUCB bandit (Algorithm 1, line 12) and the trust score tau_k (line 13), which in turn sets alpha_k (Eq. (7)). If D_val overlaps in class content with the OOD stress set of Table 1, then the controller's exploration and the trust-based blending are partially optimized on the test distribution, making the OOD gains circular. The statement that D_val is 'strictly isolated from the clients' private local data' does not establish disjointness from the OOD test. The manuscript must define D_val, report its class distribution, and demonstrate that its classes are disjoint from the OOD test classes, or argue convincingly why any overlap would not affect the conclusions.
  3. [IV.B.2, Table 3] The headline per-client improvements are not backed by significance testing. The reported relative gains of 7.66% DR and 5.08% F1 correspond to Clients 4-6 compared with FedALA, but the confidence intervals overlap heavily (e.g., Client 4 F1: FBID 0.682+/-0.054 vs FedALA 0.650+/-0.041; DR: 0.520+/-0.061 vs 0.483+/-0.045), and results are averaged over only three seeds. Without paired tests or effect sizes, the observed differences could be within run-to-run noise. Please report per-client significance tests or explicitly state the variability caveat, and clarify whether the headline numbers are selected post hoc.
minor comments (4)
  1. [IV.B.2] The text says FBID improves F1 and DR 'by up to 3.20% and 3.70%' but does not state whether these are absolute or relative; the abstract uses 'relative.' Please align the terminology across the paper.
  2. [Table 2] The text states that all frameworks maintain near-zero false positive rates, but the table does not report FPR. Please include the FPR values or remove the claim.
  3. [IV.B.3] Figure 2 is referenced in the discussion of alpha trajectories but does not appear in the manuscript text available for review. Please ensure the figure is included and legible.
  4. [II.A] The description of D_val as 'curated from public threat intelligence corpora' is too vague to assess its suitability as a neutral evaluation set. Please provide details on its size, class distribution, and any preprocessing, or explicitly state that these details are omitted.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the FBID derivation and evaluation are self-contained; the OOD split inconsistency is a benchmark-validity concern, not a circular reduction.

full rationale

I walked the claimed derivation chain: the server-side LinUCB controller, trust-based blending, and reward function. The personalized model is w_bar_k = alpha_k w_G + (1-alpha_k) w_k (Eq. 1/5), with alpha_k = 1 - tau_k updated from a z-normalized sigmoid of the reward (Eq. 6-7). The reward (Eq. 3-4) uses validation AUC improvement, local F1, and a drift penalty. These are algorithm-internal feedback signals; none of them is the reported OOD DR/F1 on the held-out stress set. The reported OOD numbers are computed after training on the 20,263-sample Table 1 set, which is not used in the optimization loop, so the headline improvements are not fitted inputs renamed as predictions. The cited techniques (LinUCB [12], Ditto [7], APFL [8], FedALA [9], CICIoT2023 [15]) are external prior work, not self-citations that carry the argument. I also checked for a self-definitional or ansatz-via-citation step and found none: the paper explicitly describes the composition and its credit-assignment motivation. One non-circular concern is that Section IV.A.1's claim that the OOD set contains 'novel attack types that are entirely absent from the clients' training data' is internally inconsistent with the listed training profiles (e.g., volumetric DoS/DDoS for Nodes 1-3 and DNS/ARP spoofing/brute force for Nodes 4-6, all of which appear in Table 1). This would weaken the benchmark's validity, but it is a data-split leak, not a case where a prediction is equivalent to its inputs by construction. It therefore does not raise the circularity score.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The method rests on several unstated premises: a representative and leakage-free server validation set, a linear reward model for the bandit, and the crucial assumption that the stress test is out-of-distribution. That last assumption is contradicted by the paper's own dataset descriptions.

free parameters (4)
  • reward weights λ1, λ2, λ3 = 1.0, 0.5, 0.01
    Chosen by hand in Section IV.A.2 to balance AUC improvement, local F1, and drift penalty; the paper states λ3 is a normalizing constant. These weights control the bandit reward and thus personalization.
  • trust decay β = 0.85
    Chosen in Section IV.A.2; controls the EMA update of trust scores and hence alpha.
  • action set A = {1, 2, 3, 5} epochs
    Discrete training budget tiers chosen in Section III.A.2; the upper bound is justified by prior FL literature but the specific set is a design choice.
  • server validation set D_val = unspecified size and content
    The paper requires a compact server-side validation set to compute rewards (Sections II.A, III.A.2) but never describes its size, source, or class distribution, so its representativeness and possible overlap with the OOD test set are uncontrolled choices.
assumptions (4)
  • ad hoc to paper The OOD stress test set contains attack classes absent from all clients' training data.
    Invoked in Section IV.A.1 to motivate the OOD evaluation, but contradicted by the training profile descriptions in the same section, which include DoS/DDoS TCP, SYN, UDP, ICMP floods and DNS/ARP spoofing/brute force that also appear in the OOD set.
  • domain assumption The server has a validation set D_val that is representative of future OOD attacks and disjoint from both client data and the test set.
    The method's reward and trust signals depend on D_val (Sections II.A, III.A.2); the paper does not establish this representativeness or disjointness.
  • standard math LinUCB assumes a linear relationship between the 6-dimensional context vector and the expected reward for each arm.
    The bandit selects epoch budgets via LinUCB in Section III.A.1; the linear reward model is a standard assumption of the algorithm.
  • domain assumption The convex combination of global and local models preserves global attack detection knowledge.
    Equation (1) and the trust-based blending in Section III.B rely on this assumption; it is plausible but not proven.
invented entities (1)
  • trust score τ_k
    purpose: Controls the blending coefficient alpha between the global and local models, pulling low-trust clients toward the global model.
    The trust score is an exponential moving average of sigmoid-transformed, z-normalized rewards (Equation 6). It is an algorithmic variable introduced by the paper, with no external falsifiable handle beyond the paper's own experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FBID: Adaptive Personalized Federated Learning for Robust Out-of-Distribution Attack Detection in IoT Networks." pith.science (2026). https://pith.science/paper/FLIATBUE

@misc{pith2026260804073,
  author       = {Pith},
  title        = {Pith review of: FBID: Adaptive Personalized Federated Learning for Robust Out-of-Distribution Attack Detection in IoT Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FLIATBUE}},
  note         = {Machine review of arXiv:2608.04073}
}
read the original abstract

Personalized Federated Learning (PFL) has emerged as a promising solution for intrusion detection in heterogeneous IoT environments, as it can improve local adaptation under highly Non-Independent and Identically Distributed (non-IID) data distributions. However, existing PFL methods often rely on client-side self-adjustment, which may lead to over-personalization and substantial degradation in out-of-distribution (OOD) attack detection. In this paper, we propose Federated Bandit Intrusion Detection (FBID), a novel adaptive PFL framework to address this limitation through server-side personalization control. In particular, FBID employs a contextual multi-armed bandit at the server to dynamically regulate each client's local training intensity according to its observed behavior and update quality. Moreover, FBID introduces a trust-based blending mechanism to derive client-specific interpolation coefficients between the global and local models, thereby preserving global attack-detection knowledge while still allowing beneficial local specialization. Through extensive experiments on the CICIoT2023 dataset under heterogeneous client distributions and OOD stress-test settings, we show that FBID improves individual client OOD Detection Rate (DR) by up to 7.66% and F1-Score (F1) by up to 5.08% (relative) over the strongest stable baseline, while also improving robustness to previously unseen attack classes.

Figures

Figures reproduced from arXiv: 2608.04073 by the authors.

Figure 1
Figure 1. The FBID framework overview. described in Section III-B. The global training objective is formulated as: min {wk}K k=1, wG X K k=1 nk N Lk( ¯wk), (2) where N = PK k=1 nk is the total number of training samples, Lk(·) denotes the local empirical loss on Dk, and αk are fixed server-assigned parameters rather than optimization variables. Here, w k implicitly depends on the server-assigned budget Ek, as the local model … view at source ↗
Figure 2
Figure 2. Trajectory of α over the first 75 communication rounds. converge toward lower αk values (approximately 0.27), indi￾cating that the server permits stronger local specialization for these nodes. In contrast, clients with more skewed or benign￾heavy local datasets (Clients 4–8) converge toward higher αk values (approximately 0.73), which keeps their personalized models naturally closer to the global model to preserve r… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 14 canonical work pages

  1. [1]

    Explainable intrusion detection for cyber defences in the internet of things: Opportunities and solutions,

    N. Moustafa, N. Koroniotis, M. Keshk, A. Y . Zomaya, and Z. Tari, “Explainable intrusion detection for cyber defences in the internet of things: Opportunities and solutions,”IEEE Communications Surveys & Tutorials, vol. 25, no. 3, pp. 1775–1807, 2023

    2023

  2. [2]

    Surveying trust-based collaborative intrusion detection: State-of-the-art, challenges and future directions,

    W. Li, W. Meng, and L. F. Kwok, “Surveying trust-based collaborative intrusion detection: State-of-the-art, challenges and future directions,” IEEE Communications Surveys & Tutorials, vol. 24, no. 1, pp. 280–305, 2022

    work page 2022

  3. [3]

    A survey on IoT intrusion detection: Federated learning, game theory, social psychology, and explainable AI as future directions,

    S. Arisdakessian, O. A. Wahab, A. Mourad, H. Otrok, and M. Guizani, “A survey on IoT intrusion detection: Federated learning, game theory, social psychology, and explainable AI as future directions,”IEEE Inter- net of Things Journal, vol. 10, no. 5, pp. 4059–4092, 2022

    work page 2022

  4. [4]

    Federated learning for internet of things: A compre- hensive survey,

    D. C. Nguyen, M. Ding, P. N. Pathirana, A. Seneviratne, J. Li, and H. Vincent Poor, “Federated learning for internet of things: A compre- hensive survey,”IEEE Communications Surveys & Tutorials, vol. 23, no. 3, pp. 1622–1658, 2021

    work page 2021

  5. [5]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahanet al., “Communication-efficient learning of deep networks from decentralized data,” inProc. Int. Conf. Artif. Intell. Statist. (AIS- TATS), 2017, pp. 1273–1282

    work page 2017

  6. [6]

    Personalized federated learning for intelligent IoT applications: A cloud-edge based framework,

    Q. Wu, K. He, and X. Chen, “Personalized federated learning for intelligent IoT applications: A cloud-edge based framework,”IEEE Open Journal of the Computer Society, vol. 1, pp. 35–44, 2020

    work page 2020

  7. [7]

    Ditto: fair and robust federated learning through personal- ization,

    T. Liet al., “Ditto: fair and robust federated learning through personal- ization,” inProc. Int. Conf. Mach. Learn. (ICML), 2021, pp. 6357–6368

    work page 2021

  8. [8]

    Adaptive personalized federated learning,

    Y . Deng, M. M. Kamani, and M. Mahdavi, “Adaptive personalized federated learning,”arXiv:2003.13461, 2020

    arXiv 2003

Show all 17 references
  1. [9]

    FedALA: adaptive local aggregation for personalized federated learn- ing,

    J. Zhang, Y . Hua, H. Wang, T. Song, Z. Xue, R. Ma, and H. Guan, “FedALA: adaptive local aggregation for personalized federated learn- ing,” inProc. AAAI Conf. Artif. Intell., vol. 37, no. 9, 2023, pp. 11 237– 11 244

    2023

  2. [10]

    Lower bounds and optimal algorithms for personalized federated learning,

    F. Hanzely, S. Hanzely, S. Horv ´ath, and P. Richt´arik, “Lower bounds and optimal algorithms for personalized federated learning,” inAdvances in Neural Information Processing Systems, vol. 33, 2020, pp. 2305–2315

    2020

  3. [11]

    Adapt to adaptation: Learning personalization for cross-silo federated learning,

    J. Luo and S. Wu, “Adapt to adaptation: Learning personalization for cross-silo federated learning,” inProceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, 2022, pp. 2166–2172. [Online]. Available: https://doi.org/10.24963/ijcai.2022/301

    2022 doi

  4. [12]

    A contextual-bandit approach to personalized news article recommendation,

    L. Li, W. Chu, J. Langford, and R. E. Schapire, “A contextual-bandit approach to personalized news article recommendation,” inProc. Int. World Wide Web Conf. (WWW), 2010, pp. 661–670

    2010

  5. [13]

    FedEff: efficient federated learning with optimal local epochs for heterogeneous clients,

    K. Narmadha and P. Varalakshmi, “FedEff: efficient federated learning with optimal local epochs for heterogeneous clients,”Scientific Reports, vol. 15, no. 1, p. 38860, 2025

    2025

  6. [14]

    FedDdrl: Federated double deep reinforcement learning for heterogeneous IoT with adaptive early client termination and local epoch adjustment,

    Y . J. Wong, M.-L. Tham, B.-H. Kwan, and Y . Owada, “FedDdrl: Federated double deep reinforcement learning for heterogeneous IoT with adaptive early client termination and local epoch adjustment,” Sensors, vol. 23, no. 5, p. 2494, 2023

    2023

  7. [15]

    CICIoT2023: a real-time dataset and benchmark for large- scale attacks in IoT environment,

    E. C. P. Neto, S. Dadkhah, R. Ferreira, A. Zohourian, R. Lu, and A. A. Ghorbani, “CICIoT2023: a real-time dataset and benchmark for large- scale attacks in IoT environment,”Sensors, vol. 23, no. 13, p. 5941, 2023

    2023

  8. [16]

    PFLlib: a beginner-friendly and comprehensive personalized federated learning library and benchmark,

    J. Zhang, Y . Liu, Y . Hua, H. Wang, T. Song, Z. Xue, R. Ma, and J. Cao, “PFLlib: a beginner-friendly and comprehensive personalized federated learning library and benchmark,”J. Mach. Learn. Res., vol. 26, no. 50, pp. 1–10, 2025

    2025

  9. [17]

    Multivariate stochastic approximation using a simultane- ous perturbation gradient approximation,

    J. C. Spall, “Multivariate stochastic approximation using a simultane- ous perturbation gradient approximation,”IEEE Trans. Autom. Control, vol. 37, no. 3, pp. 332–341, 1992

    1992

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.