{"id":"803a092a-f937-4f15-8fe7-3c2f68fd55b4","arxiv_id":"2507.09805","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Two graph-based averaging rules for federated traffic forecasting achieve slightly lower RMSE than recent graph-based federated baselines on METR-LA and PEMS-BAY.","lead":"This paper proposes two simple graph-weighted averaging rules for federated traffic forecasting, where nearby road sensors influence each other's model updates. The methods are computationally light and slightly reduce prediction error on two public traffic datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Baselines FMTL and CNFGNN are taken from the original papers, not re-run under the 5-round protocol used for the proposed methods; the claimed 1.9–8.1% advantage may reflect a mismatch in training budget rather than algorithmic superiority.","rationale":"The reader's weakest assumption focused on the fidelity of the road-network graph as a proxy for parameter similarity. That is a legitimate secondary concern, but the most load-bearing issue is the evaluation protocol: the headline improvement over CNFGNN and FMTL rests on numbers lifted from a different experimental setup. The paper itself flags this in the Table 1 footnote, yet Section 6 treats the comparison as decisive. If the baselines were re-run under the same 5-round budget, the reported margin could easily vanish or invert, especially on METR-LA where the gap is below 0.1%. No error bars or seed variation are reported, so even the direction of the effect is not established. I would keep the conditional verdict because the underlying idea is simple and testable, but the condition should now be explicitly: demonstrate the advantage under a matched optimization budget and with repeated runs. This concern is distinct from the graph-construction issue; it attacks the central comparative claim directly. A single controlled re-run experiment would resolve it, so rejection is premature, but acceptance is not warranted on the current evidence.","tokens_in":9259,"tokens_out":5915,"duration_ms":69408,"concrete_test":"Re-run FMTL and CNFGNN in the authors' own pipeline with exactly the Section 5.3 protocol (Adam lr=1e-3, GRU hidden=100, batch=128, local epochs=3, client-server rounds=5), for CNFGNN including its server-side GCN but keeping communication rounds at 5; then run all methods with multiple seeds (e.g., 5) and report mean±std RMSE. If CNFGNN/FMTL under this protocol achieve RMSE no worse than the proposed methods, or if the proposed methods' advantage disappears when total gradient updates are matched, the Section 6 claim fails. This single experiment distinguishes a genuine algorithmic gain from a training-budget artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section 6 is that both proposed methods 'outperform all baselines across both datasets, with improvements ranging from 1.9% to 8.1% over the next-best method (CNFGNN).' The load-bearing assumption is that this comparison is apples-to-apples. It is not. Table 1's footnote says FMTL and CNFGNN results are '* Results as provided by [15]'; only GRU (centralized), GRU (local), and GRU + FedAvg were re-run. Section 5.3 fixes the protocol for the proposed methods at 'local epochs to 3, and client-server rounds to 5,' with Adam lr=1e-3 and hidden size 100. CNFGNN, as originally reported in [15], is a split-learning system with a server-side GCN trained over many communication rounds; its published RMSE (3.822 on PEMS-BAY, 11.487 on METR-LA) reflects a different optimization budget. Comparing a 5-round GRU aggregation rule to a converged CNFGNN number cannot support the 'outperform' claim: the ranking could reverse if baselines were limited to 5 rounds or if the proposed methods were given the full budget of [15]. The problem is compounded by the absence of repeated runs; the METR-LA differences are about 0.1% (11.473 vs 11.487), well within typical seed-to-seed variation for GRU training. Thus the claimed 1.9–8.1% improvement range is not statistically established and may be a comparison artifact.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two lightweight graph-aware federated averaging methods, GraphFedAvg and MPFedAvg, for traffic forecasting. In GraphFedAvg, the server replaces FedAvg's uniform averaging with a normalized averaging over each client's graph neighbors; in MPFedAvg, a label-propagation-style update blends each client's parameters with a degree-normalized neighborhood average controlled by a scalar α. The authors argue that these methods capture spatial dependencies between clients without the computational cost of full server-side GNN training. The experimental section compares the methods against GRU-based centralized, local, and FedAvg baselines, as well as FMTL and CNFGNN, on METR-LA and PEMS-BAY, reporting RMSE values. The central claim in Section 6 is that the proposed methods outperform all baselines on both datasets with improvements of 1.9% to 8.1% over CNFGNN.","tokens_in":9640,"tokens_out":3727,"duration_ms":38750,"significance":"If the experimental claims were fully supported, the paper would offer a simple and computationally attractive alternative to graph-neural-network-based federated aggregation for spatio-temporal forecasting. The aggregation rules in Sections 4.1 and 4.2 are clearly specified and easy to implement, and they are a reasonable extension of FedAvg that exploits a known client graph. The paper also correctly identifies that existing server-side graph approaches introduce substantial complexity. However, the current experimental validation is not sufficient to establish the central claim: the comparison against the two strongest baselines relies on numbers taken from a different experimental protocol, no repeated runs are reported, the metric set is incomplete, and one variant actually performs worse than CNFGNN on METR-LA in Table 1. The contribution is potentially useful, but the evidence presented here is not yet convincing.","major_comments":[{"comment":"The central claim that 'our proposed methods outperform all baselines across both datasets' is directly contradicted by Table 1 for MPFedAvg (1L) on METR-LA: the reported RMSE is 11.489, which is higher than the CNFGNN value of 11.487. Even if this small difference were within noise, the sentence as written is false for at least one proposed variant. The claim should be restricted to the variants that actually improve over CNFGNN, and the comparison needs statistical support.","section":"Section 6, Table 1"},{"comment":"The comparison against FMTL and CNFGNN is not apples-to-apples. The footnote to Table 1 states that FMTL and CNFGNN results are '* Results as provided by [15]', and only GRU (centralized), GRU (local), and GRU + FedAvg were re-run. Section 5.3 fixes the protocol for the proposed methods to 3 local epochs and 5 client-server rounds with Adam lr=1e-3. The published CNFGNN and FMTL numbers in [15] come from a different training budget and protocol. Therefore the claimed 1.9%–8.1% improvement over CNFGNN may be a comparison artifact rather than a property of the proposed aggregation rules. The authors must either re-run these baselines under the identical protocol or substantially qualify the claim.","section":"Section 5.3 and Table 1 footnote"},{"comment":"The experimental evidence lacks statistical grounding. Only single RMSE values are reported, with no error bars, standard deviations, or repeated seeds, even though Section 5.1 says three metrics (MAE, MAPE, RMSE) are used and only RMSE is shown. The METR-LA differences between the best proposed variant (11.473) and CNFGNN (11.487) are about 0.1%, which is well within typical seed-to-seed variation for GRU training; without repeated runs this cannot support an 'outperform' claim. The authors should report mean and variance over multiple seeds for all methods, including the re-run baselines.","section":"Table 1 and Section 5.1"},{"comment":"There is an internal inconsistency in the baseline narrative. The paper states that centralized GRU (4.172 on PEMS-BAY) 'serves as an upper bound' and that local training shows weaker results, but Table 1 reports GRU (local) at 4.152, which is better than centralized GRU on PEMS-BAY. This contradiction suggests either a protocol difference, a typo, or an issue with how the baselines were run, and it must be explained before the baseline discussion can be trusted.","section":"Section 6, first two paragraphs"},{"comment":"The two hyperparameters of the proposed methods, α=0.8 for MPFedAvg and the propagation depth L for both methods, are fixed by hand with no sensitivity analysis or validation-curve evidence. Since the reported gains are small on METR-LA, the possibility that these choices were tuned on the test set is a real concern. The authors should provide a sensitivity study over α and L, and an ablation with an alternative graph (e.g., random or learned) to justify that the road-network adjacency, not merely the averaging operation, is responsible for the improvements.","section":"Sections 4.1, 4.2 and 5.3"}],"minor_comments":[{"comment":"There is a typo in the sentence 'each client maintains its own local data (spatial and temporal) and and trains its local models'; the duplicated 'and' should be removed.","section":"Section 2, 'and and'"},{"comment":"The name 'LPFedAvg' appears in the sentence 'For the Graph Message Passing Aware (LPFedAvg) model we set α = 0.8', but the method is called MPFedAvg in Sections 4.2, 6, and the abstract. The naming should be made consistent.","section":"Section 5.3"},{"comment":"The adjacency matrix A is defined as binary (A_ij ∈ {0,1}), while Section 5.1 says the adjacency is constructed with a thresholded Gaussian kernel over road-network distances, which typically produces real-valued weights. Please clarify whether the aggregation uses the binary adjacency or the weighted kernel values.","section":"Section 3"},{"comment":"The paper says experiments evaluate MAE, MAPE, and RMSE, but Table 1 reports only RMSE. Either the other metrics should be reported or the text should be adjusted to state that RMSE is the primary reported metric.","section":"Section 5.1 and Table 1"},{"comment":"The conclusion repeats the '1.9% to 8.1%' improvement claim without acknowledging the caveat that FMTL and CNFGNN numbers are taken from a different protocol. Please qualify this claim in the conclusion as well.","section":"Section 7"}],"recommendation":"major_revision","confidential_remarks":"This is a borderline case. The core idea is simple and plausibly useful, but the experimental validation is currently too weak for the claims made. The 'results as provided by [15]' footnote means the headline comparison is not a controlled experiment. I would not reject outright because the problem is fixable: re-run FMTL and CNFGNN under the same protocol, add repeated runs with error bars, and correct the internal inconsistencies. If the authors cannot re-run those baselines, the claims must be substantially narrowed. I would also encourage the authors to consider whether a journal venue is appropriate for a paper whose contribution is a small modification to FedAvg, given the current evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: this paper proposes two simple graph-aware aggregation rules for federated traffic forecasting, and the math is fine, but the empirical claim that they beat CNFGNN and FMTL is not supported by the experiments as designed. The main problem is a mismatch in training budget.\n\nWhat's new: GraphFedAvg and MPFedAvg are exactly what they sound like — normalized adjacency averaging and label-propagation-style mixing applied to client model parameters at the server. That's a clean, lightweight idea, and the paper positions it correctly against heavier server-side GNN models like CNFGNN. The derivations in Sections 4.1 and 4.2 are straightforward and correct. The related work is thorough and gives credit to the closest mechanisms (FMTL's adjacency regularization, GFIoTL's graph filter). No equation-level circularity; the hyperparameters (alpha=0.8, propagation depth L) are chosen by hand, which is a minor overfitting risk.\n\nThe soft spots are in the experiments. The headline number — 1.9% to 8.1% over CNFGNN — is built on a comparison that is not apples-to-apples. The CNFGNN and FMTL rows in Table 1 are taken from reference [15]; only GRU-centric settings were re-run. The proposed methods use 5 client-server rounds with 3 local epochs, while CNFGNN's published numbers reflect a full-scale split-learning training schedule. You can't claim algorithmic superiority over a converged number when your method gets a lighter budget and the baseline never gets your lighter budget. The METR-LA differences are also tiny: 11.473 vs 11.487, well within seed-to-seed noise. There are no error bars or repeated runs, and RMSE is the only reported metric even though the paper promises MAE and MAPE. On top of that, Table 1 has local GRU (4.152) beating centralized GRU (4.172) on PEMS-BAY, which contradicts the paper's own narrative that centralized training is the upper bound. That's not disqualifying — it can happen with a poorly tuned central baseline — but it signals the experiments are not carefully validated.\n\nNet: the aggregation idea is worth knowing and could be useful for resource-constrained settings, but the evaluation as it stands does not establish the claimed improvements. A serious referee should ask for re-running all baselines under the same protocol, multiple seeds, error bars, and sensitivity to the graph construction. I'd send it to review before rejecting: the contribution is clear, the math is correct, and the gap is in experimental rigor, which is fixable. For a reading group, it's a good example of why baseline budgets matter.","headline":"A clean lightweight aggregation idea, but the comparison to heavier baselines is not apples-to-apples; the claimed 1.9-8.1% gains are not established by the current experiments.","tokens_in":10118,"tokens_out":1996,"would_cite":false,"duration_ms":19611,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two lightweight graph-aware aggregation rules for federated traffic forecasting beat FedAvg, FMTL, and CNFGNN on METR-LA and PEMS-BAY, the paper reports.","keywords":["federated learning","traffic forecasting","graph-based aggregation","federated averaging","label propagation","spatio-temporal prediction","METR-LA","PEMS-BAY"],"falsifier":"A degree-preserving random permutation of the adjacency matrix, rerun through both aggregation rules, would settle the central mechanism: if RMSE does not rise substantially, the road graph's spatial structure is not what produces the reported gains.","tokens_in":9097,"feed_emoji":"🚗","tokens_out":10131,"duration_ms":105382,"temperature":0.7,"pith_summary":"The paper tries to establish that server-side graph-aware aggregation for federated traffic forecasting does not need a graph neural network: two simple weighted-averaging rules that use the road-network graph can outperform standard federated averaging and more complex graph-based methods. On the METR-LA and PEMS-BAY benchmarks, the authors report that GraphFedAvg and MPFedAvg reach lower root mean squared error than FedAvg, FMTL, and CNFGNN, with improvements of 1.9% to 8.1% over the next-best method, while adding only a matrix multiplication to the server's aggregation step. If the claim holds, resource-constrained federated systems could capture spatial dependencies between sensors almost for free, without training deep graph models on the server.","feed_headline":"Road-neighbor averaging beats graph-neural federated baselines","feed_subtitle":"Two FedAvg-style server rules add spatial weighting and outscore deeper baselines on METR-LA and PEMS-BAY.","key_machinery":"The mechanism is a server-side replacement of uniform averaging by graph propagation over the road-network adjacency matrix $A$, augmented with self-loops as $\\tilde{A}=A+I$. GraphFedAvg iterates $X^{(\\ell+1)}=\\tilde{D}^{-1}\\tilde{A}X^{(\\ell)}$ for $L$ steps, where $\\tilde{D}$ is the degree matrix of $\\tilde{A}$; this makes each client's updated parameters a degree-normalized average of parameters from its $L$-hop neighbourhood. MPFedAvg instead iterates the label-propagation-style rule $X^{(\\ell+1)}=\\alpha\\,\\tilde{D}^{-1/2}\\tilde{A}\\tilde{D}^{-1/2}X^{(\\ell)}+(1-\\alpha)X^{(\\ell)}$, with $\\alpha=0.8$, blending a symmetrically normalized neighbor average with each client's own parameters. Both rules are linear in the parameter matrix, so they keep the aggregation step close to FedAvg in cost while letting information flow across the sensor graph.","core_discovery":"The central claim, stated by the authors in the results section, is that replacing the uniform average of FedAvg with a neighbourhood average over the road graph yields the best forecasting accuracy among all compared methods on both datasets. The best single-layer results are RMSE 3.733 on PEMS-BAY for MPFedAvg and 11.473 on METR-LA for GraphFedAvg, against 3.822 and 11.487 for CNFGNN, the strongest baseline. The authors conclude that a single propagation step captures the essential spatial dependencies, that a second step adds almost nothing, and that the gain comes from preserving inter-client dependencies without server-side GNN training.","pith_inferences":["An implication the authors leave implicit is that the graph's role may be regularization rather than new information: a degree-preserving random rewire of the adjacency matrix should erase most of the reported gain if the road structure is doing the work.","A testable extension beyond the paper is to replace the static road graph with a graph learned from parameter similarity each round; this would show whether the gains require physical proximity or merely a better averaging weight.","Because the aggregation rules are agnostic to the local model, the same gains may transfer to clients using architectures other than GRUs; the current experiments do not vary the client model."],"forward_implications":["If the reported results hold, a single propagation step is enough for most of the benefit, so the one-layer variants offer the best accuracy-to-cost trade-off.","The server can skip GNN training entirely; the entire graph-aware component is a few sparse matrix multiplications over client parameters.","The privacy properties of standard federated learning are preserved, because the server still receives only model updates, never raw traffic readings.","The same aggregation recipe is directly reusable in any federated spatio-temporal task where a meaningful client-similarity graph exists."],"supporting_citations":[{"why":"supplies the benchmark task setup, the adjacency-matrix construction, the CNFGNN baseline, and the baseline RMSE values the paper compares against.","marker":"[15]"},{"why":"defines the FedAvg aggregation that the proposed methods extend and that serves as the main uniform-averaging baseline.","marker":"[14]"},{"why":"provides the label-propagation update that MPFedAvg adapts into a symmetric normalized neighbor average.","marker":"[27]"},{"why":"is the original source of the METR-LA and PEMS-BAY traffic datasets used in the experiments.","marker":"[11]"},{"why":"defines the FMTL baseline with cluster regularization that the methods must beat.","marker":"[18]"},{"why":"provides the GRU encoder-decoder architecture used as each client's local temporal model.","marker":"[2]"}],"fun_headline_variants":["Simple graph-weighted FedAvg tops GNN baselines in traffic","Neighborhood averaging beats GNN-heavy federated learning","One-hop graph weighting wins over deep GNN federated models","Lightweight FedAvg variant outperforms graph neural nets","Graph-aware FedAvg: simple aggregation, better traffic forecasts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the sensor-distance road graph is a faithful map of which clients' model parameters should be averaged together; if the graph does not track parameter similarity, the neighbourhood weighting can only distort the global model.","fun_headline_variants_meta":{"raw":{"variants":["Simple graph-weighted FedAvg tops GNN baselines in traffic","Neighborhood averaging beats GNN-heavy federated learning","One-hop graph weighting wins over deep GNN federated models","Lightweight FedAvg variant outperforms graph neural nets","Graph-aware FedAvg: simple aggregation, better traffic forecasts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000195,"raw_usage":{"total_tokens":1337,"prompt_tokens":908,"completion_tokens":429,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":524,"completion_tokens_details":{"reasoning_tokens":348}},"tokens_in":524,"tokens_out":429,"duration_ms":4692,"temperature":1.0,"reasoning_tokens":348,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:46:25.880378+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A degree-preserving random permutation of the adjacency matrix, rerun through both aggregation rules, would settle the central mechanism: if RMSE does not rise substantially, the road graph's spatial structure is not what produces the reported gains.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the label-propagation update that MPFedAvg adapts into a symmetric normalized neighbor average."}],"review_version":1}