{"id":"a4067798-d493-4754-a284-616942db43e0","arxiv_id":"2505.01186","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"DARCS, a reliability-based client selection and anomaly detection framework, keeps hierarchical federated learning in vehicular networks within 2-3% of attack-free accuracy and reduces convergence time under noise and gradient ascent attacks.","lead":"This paper proposes a defense framework called DARCS that filters unreliable or malicious vehicle updates during hierarchical federated learning, using Z-score, cosine similarity, adaptive thresholds, and cross-cluster checks. The paper shows in simulations that the framework keeps model accuracy close to the attack-free baseline and speeds convergence compared to simpler defenses.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DARCS relies on an unflagged labeled validation dataset at the EPC and every CH; in standard federated learning this data is unavailable, so the claimed defense rests on an oracle assumption.","rationale":"The reader's weakest-assumption analysis identifies the same load-bearing issue: the framework requires a labeled validation dataset at the EPC and every CH to compute Historical Accuracy, which feeds reliability scores, adaptive thresholds, and weighted averaging. This is the correct primary concern because it is not a mere implementation detail; it is the foundation of the proposed defense. If D_val is unavailable, the novel components of DARCS cannot operate as described, and the paper would reduce to a benchmark-level defense. I considered the mismatch between the abstract's 17.1%/8.7% figures and Table I, but that is a reporting/verification issue; the labeled-validation assumption is more fundamental because it determines whether the proposed defense is even implementable in the intended FL setting. The paper neither lists D_val as an assumption nor discusses its provenance or privacy implications, despite using it in Eq. 10-11 and Algorithm 1 lines 20-24. A straightforward ablation removing this oracle would settle whether the claimed gains survive under standard FL data availability. Since the reader already marked the paper CONDITIONAL on related grounds, my stress-test does not change the verdict; it strengthens the reason for conditioning acceptance on resolving or explicitly justifying the D_val assumption.","tokens_in":17771,"tokens_out":4449,"duration_ms":46511,"concrete_test":"Run a controlled ablation of the 25-vehicle, 100-meter, 1-hop combined noise-plus-gradient-ascent scenario reported in Table I (rows 6-8) with D_val removed: set Accuracy Weight = 0 in Eq. 15, disable the adaptive threshold condition in Eq. 16, keep all other DARCS mechanisms unchanged, and report convergence time and final accuracy across at least 5 seeds. If the resulting accuracy loss is no longer within 2-3% of the no-attack baseline and convergence time no longer beats the combined Z-score + cosine benchmark, the headline claim is contingent on the labeled-validation assumption.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central defense mechanism depends on Historical Accuracy (Eq. 10-11), computed by evaluating each vehicle's updated model on a labeled validation dataset D_val at the CH and EPC. Reliability Score (Eq. 15), adaptive thresholding (Eq. 16, Algorithm 1 lines 26-27, Algorithm 2 lines 32-33), and weighted gradient averaging (Eq. 17 and Eq. 19) all require this score. The paper never flags D_val as an assumption: Algorithm 1 lines 20-24 and Algorithm 2 lines 26-31 simply treat 'Accuracy of Contribution' as available. In standard federated learning, the server and edge aggregators do not possess labeled client data; if D_val is absent, Historical Accuracy is undefined, adaptive thresholding cannot activate, and weighted averaging degenerates to unweighted averaging of already-filtered updates. The quantitative claims of 2-3% accuracy loss and up to 17% faster convergence are therefore demonstrated only under a defender oracle that violates the usual FL privacy model. Even when D_val exists, a client with high accuracy on D_val may merely have local data similar to D_val under non-IID distributions, so using it as a trust signal can systematically favor certain clients rather than detect poisoning. Without a labeled validation set, DARCS reduces to roughly a Z-score + cosine defense with contribution-frequency and anomaly-record metrics, i.e., close to the 'combined defense' benchmark, and the claimed advantage is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DARCS, a defense framework for cluster-based hierarchical federated learning (HFL) in vehicular networks. DARCS combines Z-score and cosine-similarity anomaly detection with per-vehicle reliability scores built from historical accuracy, contribution frequency, and anomaly records; it adds an adaptive thresholding mechanism for cosine similarity, a cross-cluster consistency check, and reliability-weighted gradient averaging. The authors evaluate DARCS on an MNIST-based non-IID vehicular simulation with 20% attackers under Gaussian noise, gradient ascent, and combined attacks, in both 1-hop and 3-hop topologies, comparing against cosine-similarity-only, Z-score-only, and a combined baseline. The paper claims that DARCS limits accuracy loss to within 2–3% of the attack-free baseline and reduces convergence time by up to 17% relative to the benchmarks.","tokens_in":18119,"tokens_out":7691,"duration_ms":78575,"significance":"If the claims hold, DARCS would be a practically useful, multi-level defense for hierarchical federated learning in vehicular settings, and the paper addresses a genuine gap: existing defenses are not designed for the cluster-based HFL structure. The algorithm descriptions are unusually complete, with pseudocode for both the CH and EPC levels, and the evaluation spans multiple attack types, network sizes, transmission ranges, and convergence thresholds. However, the central quantitative claims are not reproducible from the reported tables, and a key input—labeled validation data at the aggregators—is never stated as an assumption. The value of the contribution therefore depends on whether these issues can be resolved with additional experiments and a clearer statement of the threat and system model.","major_comments":[{"comment":"Historical Accuracy and every Accuracy of Contribution computation require a labeled validation set D_val at the CH and EPC. The paper never lists this as an assumption. In standard federated learning, the aggregator does not hold labeled client data; without D_val, Eq. (10) is undefined, Eq. (15) and the adaptive-thresholding trigger cannot be evaluated, and Eqs. (17) and (19) reduce to unweighted averaging over the Z-score/cosine-filtered updates. The claimed 2–3% accuracy loss and up to 17% faster convergence are therefore demonstrated only under an unstated availability of validation data. Please state this assumption explicitly, justify its availability in the vehicular HFL setting, or provide experiments without D_val. Even when D_val exists, using per-client accuracy on a fixed D_val as a trust signal can systematically favor clients whose local data distribution resembles D_val under non-IID data, so the validity of the metric needs discussion.","section":"Section III.A.3, Eqs. (10)–(11), Algorithms 1 and 2"},{"comment":"The abstract states that under combined noise and gradient ascent in the 1-hop scenario DARCS reduces convergence time by 17.1% compared to cosine-similarity and Z-score defenses and by 8.7% over the combined defense. These percentages do not appear in the combined-attack rows of Table I; they match the noise-only rows instead (DARCS 63 vs. Z-score 76 = 17.1%, and DARCS 63 vs. combined 69 = 8.7%, at ε=0.01). For the combined-attack row at ε=0.01, the improvements are 12.2% over cosine similarity (65 vs. 74) and 7.1% over the combined defense (65 vs. 70). The 3-hop percentages (15.6% and 7.1%) also do not match a single row of Table II. Please recompute the headline numbers and cite the exact table rows, or revise the claims.","section":"Abstract and Section V vs. Tables I and II"},{"comment":"Equation (16) defines the adaptive threshold as itself minus δ. As written, this is not a mathematical definition; if it is intended as an iterative update rule, the initial value and the update schedule are missing. The algorithms do not specify how many times δ is applied per round, whether the threshold is per-vehicle state, or what happens at the floor High Threshold Down. Since adaptive thresholding is credited with 4–6% accuracy improvement in Section V, this mechanism needs a well-defined recurrence with an explicit initial condition and update trigger.","section":"Section III.A.4, Eq. (16), Algorithm 1 lines 26–27, Algorithm 2 lines 32–33"},{"comment":"The performance comparison is based on single runs. The mobility model, data partitioning, and client selection contain randomness, but no standard deviations, confidence intervals, or numbers of seeds are reported. Many of the claimed improvements are 2–7 rounds out of roughly 60–80 rounds, which may be within run-to-run variation. Please report repeated-seed experiments with error bars and a sensitivity analysis over Z Score Threshold, δ, High Threshold Up/Down, and the reliability-score weights. This is needed to support the statements that DARCS “consistently outperforms” the benchmarks and achieves performance “remarkably close” to the no-attack baseline.","section":"Section IV, Tables I and II"},{"comment":"The reliability score in Eq. (15) is a weighted sum with a negative anomaly term but no non-negativity constraint. The weighted averaging in Eqs. (17) and (19) divides by the sum of reliability scores; if any score is negative or the total sum is non-positive, the aggregation is ill-defined or the update direction can be inverted. Please specify the allowed range of reliability scores and the normalization strategy, or prove that the scores remain positive under the proposed update rules.","section":"Eqs. (15), (17), and (19)"}],"minor_comments":[{"comment":"The introduction says that Section VI presents concluding remarks, but the conclusion is actually Section V.","section":"Section I, last paragraph"},{"comment":"The sorting and selection of Selected Clients appear inside the per-CM foreach loop due to the indentation; these operations should be performed once per round, not once per CM.","section":"Algorithm 1, lines 8–9"},{"comment":"The text says the CH cosine similarity compares each CH’s update with the global model parameters, but Eq. (7) defines a temporal consistency between the current and previous aggregated updates relative to the previous global model; these descriptions should be reconciled.","section":"Algorithm 2, line 10 and surrounding text"},{"comment":"The accuracy-loss figures are inconsistent: Section IV.B says adaptive thresholding limits loss to within 1–3% of baseline, while Section IV.C and the conclusion say 2–3%; please use one consistent range.","section":"Section IV.B vs. Section IV.C"},{"comment":"The reliability-score weights are described only as “equal”; please provide the exact weight values and the accuracy/frequency/anomaly normalization in the setup table.","section":"Section IV.A"}],"recommendation":"major_revision","confidential_remarks":"The validation-data assumption is the most serious concern: if the intended setting is standard federated learning, the defense relies on information that the aggregator is not assumed to possess, and the privacy framing in the introduction makes the omission particularly noticeable. I would recommend asking the authors to either justify the validation set in the vehicular HFL architecture or to add experiments that do not use it, in addition to correcting the abstract percentages. The paper may be suitable for the journal after these points are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about arXiv:2505.01186. First, it is a serviceable integration of known defense mechanisms (Z-score, cosine similarity, reliability scoring, weighted averaging) into cluster-based hierarchical FL for vehicular networks, and the ablation study suggests each of its two new pieces (adaptive thresholding, cross-cluster consistency check) contributes a few points of accuracy. Second, the quantitative summary in the abstract is not reproducible from the tables, and the whole reliability-scoring apparatus quietly assumes the CHs and EPC have a labeled validation dataset, which is not standard FL and is never flagged as an assumption.\n\nWhat the paper does well: the system model is concrete (SUMO mobility, KAFKA streaming, 802.11p and 5G NR), the algorithms are spelled out, and the evaluation compares against several baselines. The ablation of adaptive thresholding and cross-cluster checks is exactly the right way to show that the added complexity buys something. If I take their setup at face value, DARCS does tend to converge faster than the baselines and keep accuracy within a few percent of the no-attack case.\n\nSoft spots, in order of size. The labeled validation dataset issue is the big one. Equations (10)-(11) and Algorithms 1 and 2 compute \"Accuracy of Contribution\" by running each vehicle's updated model on D_val held by the CH and EPC. If that data exists, the whole mechanism works; if not, Historical Accuracy is undefined, adaptive thresholding never fires, and the reliability score loses its accuracy component. In standard federated learning the server does not have labeled client data, so this is close to a defender oracle. The paper treats D_val as a free parameter. Even when D_val exists, using it as a trust signal under non-IID data can systematically favor vehicles whose local distribution resembles D_val rather than detecting poisoning — the stress-test note raises this and I think it is a real concern, though a secondary one.\n\nSecond, the abstract's 17.1% convergence-time reduction does not match Table I. The tables show DARCS besting Z-score or combined defenses by roughly 7-12% in most comparable rows; the 17.1% figure looks like it comes from the noise-attack column at epsilon=0.01, not the combined attack as claimed. Third, Eq. (16) is written as self-referential equality (threshold = threshold − delta); it is clearly meant as an update rule, but as printed it is wrong. Finally, there are no error bars or multiple seeds, so the accuracy curves could be noisy; this is minor for a simulation paper but worth noting.\n\nWho is this for: researchers working on HFL security or vehicular network trust, who may find the integration and ablation useful as a baseline. It is not a breakthrough, but it is not a toy either. I would not cite it myself because of the oracle assumption, but I would send it to review and let the referees push for a revision that flags D_val, fixes the abstract numbers, and either provides a privacy-preserving way to obtain validation data or narrows the claims accordingly.","headline":"Plausible integration of known defenses with a solid ablation, but the headline numbers don't reproduce and the reliability scoring rests on an unflagged labeled validation set at the server.","tokens_in":18615,"tokens_out":3428,"would_cite":false,"duration_ms":31645,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A reliability-scoring defense keeps vehicular federated learning within 2–3% of ideal accuracy under poisoning attacks.","keywords":["hierarchical federated learning","vehicular networks","model poisoning defense","anomaly detection","dynamic client selection","Gaussian noise attack","gradient ascent attack","adaptive thresholding"],"falsifier":"Withhold the labeled validation set from cluster heads and the central server, keep 20 percent of vehicles attacking from round one, and measure accuracy and convergence time; if the model stays within 3 percent of attack-free accuracy, the validation-set assumption is not load-bearing, and if it does not, the defense's central premise is exposed.","tokens_in":17577,"feed_emoji":"🛡️","tokens_out":6470,"duration_ms":62888,"temperature":0.7,"pith_summary":"This paper proposes a defense, DARCS, for hierarchical federated learning in vehicular networks, and claims it keeps the global model within 2–3% of attack-free accuracy while cutting convergence time by up to 17% compared with Z-score-only, cosine-similarity-only, and combined defenses. The defense targets two poisoning styles: Gaussian noise added to local updates, and gradient-ascent updates that push the model away from the loss minimum. It scores every vehicle on historical accuracy, contribution frequency, and anomaly history, then uses Z-score and cosine-similarity checks, adaptive thresholds, weighted averaging, and cross-cluster consistency checks to filter suspicious contributions. The result matters because vehicular federated learning is attractive for privacy and bandwidth, but an undefended hierarchy gives attackers many places to inject bad updates. If the claim holds, this is a practical recipe for keeping collaborative vehicle models reliable under persistent attack.","feed_headline":"Poisoned vehicle learning stays within 3% of ideal accuracy","feed_subtitle":"A layered reliability-and-anomaly filter beats three benchmark defenses in simulated vehicular networks.","key_machinery":"The device carrying the argument is the reliability score, a weighted combination of historical accuracy, contribution frequency, and anomaly record (Eq. 15), used both to select participating vehicles and to weight their gradients in aggregation (Eqs. 17 and 19). Two statistical detectors feed it: a Z-score on gradient norms (Eq. 4) to catch magnitude outliers typical of Gaussian noise, and a cosine similarity against the mean update (Eqs. 6 and 7) to catch directional reversals typical of gradient ascent. An adaptive threshold (Eq. 16) tightens the cosine criterion for high-accuracy vehicles, and a cross-cluster consistency check (Eqs. 8 and 9) compares each cluster's update with all others to expose coordinated attacks. These mechanisms jointly determine which updates enter the weighted average, so the defense's power rests on the accuracy of the reliability scores.","core_discovery":"The paper's central claim is that a multi-level reliability-based filter can defend hierarchical federated learning in vehicular networks against both noise injection and gradient-ascent model poisoning without sacrificing convergence speed. At each round, cluster heads and the central EPC compute reliability scores for each vehicle from its historical accuracy on a validation dataset, its contribution frequency, and its anomaly record; Z-score analysis catches updates whose norms deviate statistically, while cosine similarity catches updates whose direction disagrees with consensus. An adaptive threshold tightens cosine-similarity scrutiny for vehicles with high historical accuracy, and a cross-cluster consistency check flags clusters whose aggregated updates diverge from the rest of the network. Accepted updates are averaged with weights proportional to reliability, so trusted vehicles dominate aggregation. Simulations across 1-hop and 3-hop topologies, with 20% of vehicles attacking from round one, show DARCS holds accuracy loss to roughly 2–3% of the attack-free baseline while benchmarks lose 8–17% and sometimes fail to converge at tight thresholds.","pith_inferences":["A testable extension would drop the labeled validation dataset from the cluster heads and EPC, since ordinary federated learning has no such server-side labels; the accuracy gap would likely widen and expose how much of the defense depends on that supervision.","The same reliability-scoring structure could be adapted to other poisoning styles, such as backdoor or label-flipping attacks, but cosine similarity against consensus would be weaker against attacks designed to stay close to the benign direction.","The adaptive threshold that tightens scrutiny for high-performing vehicles could over-exclude honest vehicles after a natural data-distribution shift, since their historical accuracy would briefly drop and their updates would look anomalous."],"forward_implications":["Vehicular HFL deployments could run under sustained Gaussian-noise and gradient-ascent attacks with convergence times near the attack-free baseline rather than diverging.","The 3-hop topology inherits an extra aggregation layer that dilutes random noise, so DARCS converges slightly faster there than in 1-hop under the same attacks.","Adaptive thresholding alone recovers 4–6% of accuracy compared with static thresholds, and cross-cluster consistency checks recover a further 3–5%.","Benchmark defenses that use only Z-score, only cosine similarity, or a fixed combination will fail to converge at tight thresholds, while DARCS continues to converge."],"supporting_citations":[{"why":"Supplies the cluster-based hierarchical federated learning architecture that DARCS is built on and enhanced.","marker":"[6]"},{"why":"The cosine-similarity client-filtering defense used as a benchmark baseline.","marker":"[15]"},{"why":"The Z-score elimination defense used as a benchmark baseline.","marker":"[13]"},{"why":"Provides the anomaly-detection foundation invoked for Z-score-based outlier filtering.","marker":"[23]"},{"why":"Defines the accuracy measure used to compute each vehicle's historical accuracy on the validation dataset.","marker":"[24]"},{"why":"Defines the epsilon-threshold convergence criterion used to measure convergence time.","marker":"[27]"}],"fun_headline_variants":["New defense keeps poisoned vehicle AI near ideal accuracy","Vehicle federated learning resists attacks to within 3%","Robust FL for cars: filter blocks poison to keep 3% loss","Multi-level defense for vehicular FL curbs poisoning impact","Cluster-based FL defense trims malicious vehicle effect"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The defense assumes the central server and every cluster head hold a labeled validation dataset on which to score each vehicle's model, yet standard federated learning gives the server no such labeled data.","fun_headline_variants_meta":{"raw":{"variants":["New defense keeps poisoned vehicle AI near ideal accuracy","Vehicle federated learning resists attacks to within 3%","Robust FL for cars: filter blocks poison to keep 3% loss","Multi-level defense for vehicular FL curbs poisoning impact","Cluster-based FL defense trims malicious vehicle effect"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00062,"raw_usage":{"total_tokens":2905,"prompt_tokens":1007,"completion_tokens":1898,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":623,"completion_tokens_details":{"reasoning_tokens":1816}},"tokens_in":623,"tokens_out":1898,"duration_ms":12948,"temperature":1.0,"reasoning_tokens":1816,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:23:49.979736+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Withhold the labeled validation set from cluster heads and the central server, keep 20 percent of vehicles attacking from round one, and measure accuracy and convergence time; if the model stays within 3 percent of attack-free accuracy, the validation-set assumption is not load-bearing, and if it does not, the defense's central premise is exposed.","supporting_citations":[{"cited_title":"Defending against Data Poisoning Attacks in Federated Learning via User Elimination","cited_arxiv_id":"2404.12778","evidence_quote":"The Z-score elimination defense used as a benchmark baseline."},{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"Defines the epsilon-threshold convergence criterion used to measure convergence time."}],"review_version":1}