{"id":"b838cb27-fe7b-4ece-9f43-5ddb0e5d92bd","arxiv_id":"2608.04073","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"FBID adds a server-side contextual bandit and trust-based model blending to personalized federated learning and reports modest OOD attack detection gains on CICIoT2023, though the OOD test set appears contaminated with training classes.","lead":"This paper proposes FBID, a federated learning method that uses a server-side bandit to set each IoT client's local training budget and a trust score to blend global and local models. It reports better detection of out-of-distribution attacks on the CICIoT2023 benchmark, with per-client gains up to 7.66% in detection rate and 5.08% in F1.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The OOD stress set is not out-of-distribution: Section IV.A.1 training profiles include volumetric DoS/DDoS and spoofing/brute-force classes that Table 1 tests, so 'unseen attack' robustness is unproven.","rationale":"The reader's weakest assumption correctly identifies the same load-bearing flaw: the OOD stress test is not actually out-of-distribution with respect to the training data. The paper's Section IV.A.1 explicitly says the OOD set contains 'novel attack types that are entirely absent from the clients' training data,' yet the same section's client profile descriptions, combined with Table 1, show that several prominent OOD classes are present in the training profiles of gateways and hubs. This is not a matter of differing from external consensus; it is an internal inconsistency between the stated evaluation design and the data partition. Because the headline improvement is specifically about OOD robustness, an evaluation that cannot guarantee the stress set is OOD cannot support the claimed contribution. The D_val issue is secondary but related: without specifying its class content, the server-side reward could leak test-distribution information. No formal verification or code release is provided to independently substantiate the numbers. Given that the central empirical claim is unsupported as reported, rejection is appropriate; the method itself may be salvageable with a correctly constructed OOD split and a disclosed D_val, but the current submission does not demonstrate it.","tokens_in":9847,"tokens_out":2752,"duration_ms":26885,"concrete_test":"Rebuild the OOD evaluation by removing from Table 1 every attack class that appears in any client's training profile, specifically DoS-TCP Flood, DoS-SYN Flood, DDoS-UDP Flood, DDoS-ICMP Flood, DDoS-SlowLoris, MITM-ArpSpoofing, DictionaryBruteForce, and DNS Spoofing. Recompute FBID versus FedALA (and other baselines) on the remaining truly unseen classes (e.g., SqlInjection, CommandInjection, XSS, Backdoor, Uploading Attack, BrowserHijacking, Mirai-greeth flood). If the per-client DR/F1 gains shrink to noise or reverse, the central claim of robustness to previously unseen attacks is not supported.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is that FBID improves OOD detection on 'novel attack types that are entirely absent from the clients' training data' (Section IV.A.1). That premise is contradicted by the paper's own setup. The training partition assigns Nodes 1-3 data 'focused on volumetric DoS/DDoS (TCP, SYN, UDP, ICMP)' and Nodes 4-6 data featuring 'DNS/ARP Spoofing, Brute Force'. Table 1's OOD stress set contains DoS-TCP Flood, DoS-SYN Flood, DDoS-UDP Flood, DDoS-ICMP Flood, DDoS-SlowLoris, MITM-ArpSpoofing, DNS Spoofing, and DictionaryBruteForce. Thus the majority of the 'unseen' attack samples are present in at least some clients' training distributions, making the measured gap at least partly an in-distribution comparison rather than a demonstration of robustness to previously unseen classes. The absence of per-class or per-client overlap details means the reader cannot tell which of the reported gains, including the headline 7.66% DR and 5.08% F1 improvements, would survive on a truly disjoint class set. The unspecified server validation set D_val compounds this: if it also contains the OOD classes, the bandit reward and trust scores are partially informed by the test distribution. These are load-bearing because the abstract and conclusion rest entirely on this OOD stress-test evaluation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FBID, a personalized federated learning framework in which a server-side contextual bandit adaptively assigns per-client local training budgets and a trust-based mechanism blends global and local models into client-specific personalized models. The stated goal is to prevent over-personalization and preserve out-of-distribution (OOD) attack-detection performance in heterogeneous IoT intrusion detection. The paper evaluates FBID on the CICIoT2023 dataset with 10 non-IID clients and reports improvements in OOD Detection Rate and F1 over baselines such as FedALA, APFL, Ditto, and CBC. The core idea of server-side control over personalization is interesting, but the experimental evaluation contains a load-bearing flaw: the OOD stress test is not actually out-of-distribution relative to the training profiles described in the same section, and the server-side validation set used to drive both the bandit and the trust score is unspecified. These issues directly undermine the paper's central claims.","tokens_in":10119,"tokens_out":7099,"duration_ms":59678,"significance":"If the claims were supported, FBID would provide a practical and low-overhead mechanism for mitigating the OOD degradation that can result from client-driven personalization in federated intrusion detection. The framework's separation of optimization (epoch budget via bandit) from interpolation (blending coefficient via trust) is a reasonable design response to the credit-assignment problem, and the communication overhead is genuinely small. However, the evidence presented does not establish the headline claims because the OOD stress set overlaps with training classes and the reward used for both control loops is computed on an unspecified validation set. The method is interesting enough to warrant a major revision, but the evaluation must be redone with a truly disjoint held-out attack-class set and a fully specified validation set.","major_comments":[{"comment":"The OOD stress test is not out-of-distribution with respect to the training data as described. Section IV.A.1 states that 'this OOD set introduces novel attack types that are entirely absent from the clients' training data,' but the same section assigns Nodes 1-3 training data 'focused on volumetric DoS/DDoS (TCP, SYN, UDP, ICMP),' Nodes 4-6 data featuring 'DNS/ARP Spoofing, Brute Force,' and Nodes 9-10 a mix of DoS/DDoS and slow-rate attacks. Table 1's stress set contains DoS-TCP Flood, DoS-SYN Flood, DDoS-UDP Flood, DDoS-ICMP Flood, DDoS-SlowLoris, MITM-ArpSpoofing, DNS Spoofing, and DictionaryBruteForce. The majority of the stress-set attack classes are therefore present in at least some clients' training distributions. The paper provides no class-level or client-level overlap analysis, so the reported gains in Tables 2 and 3 cannot be attributed to robustness to previously unseen attack classes; they may reflect in-distribution or covariate-shift performance. This directly contradicts the abstract and conclusion claims about robustness to previously unseen attacks.","section":"IV.A.1, Table 1"},{"comment":"The server-side validation set D_val is unspecified and is load-bearing for both control signals. D_val is introduced in Section II.A, but its size, class composition, and provenance are never given. The reward r_k in Eq. (3) depends on delta_AUC computed on D_val (Eq. (4)), and this reward updates both the LinUCB bandit (Algorithm 1, line 12) and the trust score tau_k (line 13), which in turn sets alpha_k (Eq. (7)). If D_val overlaps in class content with the OOD stress set of Table 1, then the controller's exploration and the trust-based blending are partially optimized on the test distribution, making the OOD gains circular. The statement that D_val is 'strictly isolated from the clients' private local data' does not establish disjointness from the OOD test. The manuscript must define D_val, report its class distribution, and demonstrate that its classes are disjoint from the OOD test classes, or argue convincingly why any overlap would not affect the conclusions.","section":"II.A, III.A, III.B (Eq. (3)-(6), Algorithm 1)"},{"comment":"The headline per-client improvements are not backed by significance testing. The reported relative gains of 7.66% DR and 5.08% F1 correspond to Clients 4-6 compared with FedALA, but the confidence intervals overlap heavily (e.g., Client 4 F1: FBID 0.682+/-0.054 vs FedALA 0.650+/-0.041; DR: 0.520+/-0.061 vs 0.483+/-0.045), and results are averaged over only three seeds. Without paired tests or effect sizes, the observed differences could be within run-to-run noise. Please report per-client significance tests or explicitly state the variability caveat, and clarify whether the headline numbers are selected post hoc.","section":"IV.B.2, Table 3"}],"minor_comments":[{"comment":"The text says FBID improves F1 and DR 'by up to 3.20% and 3.70%' but does not state whether these are absolute or relative; the abstract uses 'relative.' Please align the terminology across the paper.","section":"IV.B.2"},{"comment":"The text states that all frameworks maintain near-zero false positive rates, but the table does not report FPR. Please include the FPR values or remove the claim.","section":"Table 2"},{"comment":"Figure 2 is referenced in the discussion of alpha trajectories but does not appear in the manuscript text available for review. Please ensure the figure is included and legible.","section":"IV.B.3"},{"comment":"The description of D_val as 'curated from public threat intelligence corpora' is too vague to assess its suitability as a neutral evaluation set. Please provide details on its size, class distribution, and any preprocessing, or explicitly state that these details are omitted.","section":"II.A"}],"recommendation":"major_revision","confidential_remarks":"The OOD flaw is central to the paper's contribution: the stress test is not OOD relative to the described training profiles, and the server-side validation set D_val, which drives both the bandit and the trust score, is never specified. A revision that only rewrites claims will not be sufficient; the authors need to re-run the evaluation with a truly disjoint held-out attack-class set and a fully specified D_val whose class content is shown to be disjoint from the test set. If such results cannot be provided, rejection would be appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's the quick take on arXiv:2608.04073. The method is a sensible composition, but the OOD evaluation doesn't deliver what the headline claims. FBID moves personalization control to the server via a contextual bandit that picks local epoch budgets and a trust-based EMA that sets the blending coefficient between global and local models. That decoupling is genuinely new in the PFL-for-IDS space, and it's a reasonable answer to the over-personalization problem.\n\nWhat the paper does well: the problem is clearly framed, the CBC ablation isolates the effect of server-side control, and the overhead discussion is honest. The authors also state their component techniques are standard, so they're not overselling novelty. The math is straightforward—convex combination, EMA, LinUCB—and the citations include the standard PFL and bandit references. No citation red flags.\n\nThe soft spots are serious and load-bearing. Section IV.A.1 describes the training partition: gateways get volumetric DoS/DDoS classes, hubs get DNS/ARP spoofing and brute force. The OOD test set in Table 1 includes exactly those classes (DoS-TCP/SYN/UDP/ICMP, DNS Spoofing, MITM-ArpSpoofing, DictionaryBruteForce). So the claim that the stress test introduces 'novel attack types entirely absent from training data' is simply not true. The reported gains, including the 7.66% DR headline, are at least partly in-distribution comparisons.\n\nThe second problem is D_val. The server uses this validation set both for the bandit reward and for the trust scores, but the paper never says what is in it or how it is disjoint from the test set. If D_val overlaps with the OOD classes, the controller is indirectly trained on the test distribution. That's a reproducibility and design gap the authors need to close.\n\nThird, the per-client gains in Table 3 are mostly within one standard deviation of the baselines, and with three seeds and no significance testing, the aggregate numbers are suggestive at best.\n\nWho is this for? Researchers working on PFL for intrusion detection will find the controller design useful, but the evaluation needs a genuinely disjoint OOD set, a specified D_val, and stronger statistical evidence. As submitted, the central claim is unproven.\n\nMy recommendation: send it to peer review, but the reviewers should push for a major revision on evaluation. The idea deserves the attention, but not as is.\n\nBest,\n\n[Your name]","headline":"A sensible server-side personalization controller that is undermined by an OOD evaluation whose test set overlaps the training classes, leaving the headline robustness claim unproven.","tokens_in":10669,"tokens_out":2526,"would_cite":false,"duration_ms":21972,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FBID claims a server-side bandit plus trust-based blending of global and local models lifts per-client out-of-distribution attack detection by up to 7.66% and F1 by 5.08% over stable baselines while preserving in-distribution accuracy.","keywords":["Federated Learning","Personalized Federated Learning","Intrusion Detection","IoT Security","Out-of-Distribution Detection","Contextual Multi-Armed Bandit","Non-IID Data","Model Robustness"],"falsifier":"Check the class lists: if DoS-TCP Flood, DoS-SYN Flood, DDoS-UDP Flood, DDoS-ICMP Flood, DNS Spoofing, or DictionaryBruteForce appear in any client's training profile, then the stress set is not out-of-distribution. Re-running the comparison with a stress set built from attack families excluded from all training data, and reporting the validation set's class composition, would settle whether the claimed robustness to unseen attacks is real.","tokens_in":9613,"feed_emoji":"🛡️","tokens_out":7211,"duration_ms":59794,"temperature":0.7,"pith_summary":"The paper sets out to fix a specific failure of personalized federated learning for intrusion detection: when each IoT client tunes its own model, skewed local data can over-personalize the model and blind it to attacks it has not seen. FBID moves the personalization dial to the server. A contextual bandit picks a per-client training budget, and a trust score derived from validation reward controls how much each client's final model leans on the global model. On CICIoT2023 with 10 heterogeneous clients and 21 unseen attack classes, the paper reports up to 7.66% higher per-client OOD detection rate and 5.08% higher F1 than the best stable baseline. If the result holds, it would mean server-side oversight is a cheap way to keep global attack knowledge without giving up local adaptation.","feed_headline":"Server-side bandit lifts detection of unseen attacks by 7.66%","feed_subtitle":"A trust-weighted mix of global and local models stops client overfitting from blinding federated intrusion detection.","key_machinery":"The central machinery is the server-side controller. A LinUCB contextual multi-armed bandit selects each client's local epoch budget from {1, 2, 3, 5}; the reward combines the improvement in AUC on a server-held validation set from a shadow update, the client's local F1, and a drift penalty on the scaled weight update. An exponential moving average of the sigmoid of the z-scored reward gives a trust score tau_k, and alpha_k = 1 - tau_k interpolates between global and local models: wbar_k = alpha_k w_G + (1 - alpha_k) w_k. This decouples the optimization depth from the degree of model deviation, so a single reward signal does not have to solve the credit-assignment problem of attributing poor performance to either insufficient training or excessive personalization.","core_discovery":"On the paper's own terms, the discovery is that server-side supervision of personalization fixes the out-of-distribution failure mode of client-driven personalized federated learning. The authors show that with a single per-client reward, a contextual bandit controlling only the local epoch budget and a trust-derived blending coefficient can keep personalized models close enough to the global model on low-quality or benign-skewed clients while allowing high-quality clients to specialize. In their experiments, reliable clients converge to alpha around 0.27 and skewed clients to alpha around 0.73, whereas the client-driven CBC baseline over-personalizes benign-heavy clients (missing nearly all attacks) and over-relies on the global model for information-rich gateways. FBID reports aggregated OOD F1 of 0.680 and detection rate of 0.518, versus 0.660 and 0.494 for the strongest stable baseline FedALA, with per-client relative gains up to 5.08% in F1 and 7.66% in detection rate.","pith_inferences":["The headline margin is likely sensitive to the exact composition of the stress set; a clean test would construct OOD classes by removing entire attack families from all training partitions and then measure the FBID-versus-FedALA gap.","The same server-side bandit plus trust-blending design could transfer to other federated domains with a public validation corpus, such as fraud or malware detection, where OOD robustness matters as much as local accuracy.","Because the server-side validation set is the only channel through which the server observes client behavior, the framework's behavior under different validation-set distributions is an open question; a sensitivity study over validation sources would sharpen the practical guidance.","The trust score's neutral prior of 0.5 means early rounds treat all clients equally, so the bandit's exploration phase is load-bearing for convergence speed and could be tuned per deployment."],"forward_implications":["Deploying FBID would let a federation keep the in-distribution accuracy of personalized learning while avoiding the near-complete attack-detection collapse that client-side methods show on benign-heavy clients.","The overhead is small enough for practice: one scalar epoch budget and one blending coefficient per client per round, plus a modest server-side validation set, with no change to the client model family.","The trust trajectories give operators a per-client view of update quality: reliable clients end with alpha around 0.27, while skewed clients are held near alpha 0.73, keeping them anchored to the global model.","Because the controller consumes only scalar performance signals, the same supervision mechanism could in principle be applied to sequence-based or deeper intrusion detection architectures."],"supporting_citations":[{"why":"Defines the FedAvg aggregation protocol that FBID builds on as the global training baseline.","marker":"[5]"},{"why":"Supplies the Ditto baseline, a regularization-based PFL method that fails on benign-heavy clients in the comparison.","marker":"[7]"},{"why":"Supplies the APFL baseline and the analysis that client-side alpha optimization can drive alpha toward zero, causing overfitting to local data.","marker":"[8]"},{"why":"Supplies the FedALA baseline, which is the strongest stable comparison for the headline OOD gains.","marker":"[9]"},{"why":"Provides the LinUCB algorithm used as the server-side contextual multi-armed bandit.","marker":"[12]"},{"why":"Justifies the five-epoch upper bound by showing that excessively deep local training exacerbates client drift under non-IID conditions.","marker":"[13]"},{"why":"Validates adaptive local epoch adjustment via a discrete action space in heterogeneous IoT federated settings.","marker":"[14]"},{"why":"Provides the CICIoT2023 dataset used for the non-IID client partition and the OOD stress test.","marker":"[15]"},{"why":"Provides the implementation library used to run the federated experiments.","marker":"[16]"},{"why":"Supplies the SPSA optimizer used by the CBC ablation to tune the blending coefficient client-side.","marker":"[17]"}],"fun_headline_variants":["Server-side bandit lifts unseen attack detection by 7.66%","Trust-based model mix stops IoT client overfitting","Bandit-tuned personalization improves OOD attack detection","Adaptive PFL uses bandit to curb over-personalization","FBID: server-side bandit boosts OOD detection in IoT"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim collapses if the OOD stress set is not actually out-of-distribution: Table 1's 'unseen' classes overlap with the client training profiles described in Section IV.A.1 (volumetric DoS/DDoS for gateways, DNS/ARP spoofing and brute force for hubs), and the paper does not establish that the server validation set is class-disjoint from the test set.","fun_headline_variants_meta":{"raw":{"variants":["Server-side bandit lifts unseen attack detection by 7.66%","Trust-based model mix stops IoT client overfitting","Bandit-tuned personalization improves OOD attack detection","Adaptive PFL uses bandit to curb over-personalization","FBID: server-side bandit boosts OOD detection in IoT"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00069,"raw_usage":{"total_tokens":3151,"prompt_tokens":998,"completion_tokens":2153,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":614,"completion_tokens_details":{"reasoning_tokens":2067}},"tokens_in":614,"tokens_out":2153,"duration_ms":14236,"temperature":1.0,"reasoning_tokens":2067,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:45:18.968105+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Check the class lists: if DoS-TCP Flood, DoS-SYN Flood, DDoS-UDP Flood, DDoS-ICMP Flood, DNS Spoofing, or DictionaryBruteForce appear in any client's training profile, then the stress set is not out-of-distribution. Re-running the comparison with a stress set built from attack families excluded from all training data, and reporting the validation set's class composition, would settle whether the claimed robustness to unseen attacks is real.","supporting_citations":[{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"Defines the FedAvg aggregation protocol that FBID builds on as the global training baseline."},{"cited_title":"Ditto: fair and robust federated learning through personal- ization,","cited_arxiv_id":null,"evidence_quote":"Supplies the Ditto baseline, a regularization-based PFL method that fails on benign-heavy clients in the comparison."},{"cited_title":"FedALA: adaptive local aggregation for personalized federated learn- ing,","cited_arxiv_id":null,"evidence_quote":"Supplies the FedALA baseline, which is the strongest stable comparison for the headline OOD gains."},{"cited_title":"A contextual-bandit approach to personalized news article recommendation,","cited_arxiv_id":null,"evidence_quote":"Provides the LinUCB algorithm used as the server-side contextual multi-armed bandit."},{"cited_title":"FedEff: efficient federated learning with optimal local epochs for heterogeneous clients,","cited_arxiv_id":null,"evidence_quote":"Justifies the five-epoch upper bound by showing that excessively deep local training exacerbates client drift under non-IID conditions."},{"cited_title":"FedDdrl: Federated double deep reinforcement learning for heterogeneous IoT with adaptive early client termination and local epoch adjustment,","cited_arxiv_id":null,"evidence_quote":"Validates adaptive local epoch adjustment via a discrete action space in heterogeneous IoT federated settings."},{"cited_title":"CICIoT2023: a real-time dataset and benchmark for large- scale attacks in IoT environment,","cited_arxiv_id":null,"evidence_quote":"Provides the CICIoT2023 dataset used for the non-IID client partition and the OOD stress test."},{"cited_title":"PFLlib: a beginner-friendly and comprehensive personalized federated learning library and benchmark,","cited_arxiv_id":null,"evidence_quote":"Provides the implementation library used to run the federated experiments."},{"cited_title":"Multivariate stochastic approximation using a simultane- ous perturbation gradient approximation,","cited_arxiv_id":null,"evidence_quote":"Supplies the SPSA optimizer used by the CBC ablation to tune the blending coefficient client-side."}],"review_version":1}