{"id":"7424ffee-c0eb-44a9-9385-a784c4de1728","arxiv_id":"2412.04081","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"On a Barcelona LTE dataset, federated LSTM training matches or beats centralized and per-base-station training for traffic forecasting and scores best on a hand-weighted sustainability metric.","lead":"Using real traffic data from five Barcelona base stations, this study compares federated, centralized, and per-site training of LSTM forecasters. Federated learning comes out competitive on accuracy and lowest on a composite sustainability score, but that score depends on weights chosen by the authors.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported S values in Table IV cannot be reproduced from the paper's own Eqs. (6)-(8) and Table IV inputs, undermining the key sustainability claim.","rationale":"The reader's weakest assumption identified the hand-assigned weights in the sustainability indicator S and the lack of sensitivity analysis. My stress-test found a more fundamental problem: the reported STr/SInf values in Table IV do not follow from the stated equations and the table's own inputs. This is not a disagreement with the field's consensus; it is an internal inconsistency in the paper's quantitative support for its central claim. The accuracy comparison between Federated (NRMSE 1.385) and Centralized (NRMSE 1.434) is also not significance-tested, but that gap is small and could be resolved with a paired test. The S indicator inconsistency is more decisive because the 'environmentally sustainable' framing of the paper rests entirely on Table IV. I still see value in the case study: the code is linked, 10 seeds are used, dataset accesses are equalized, and the ablation studies are informative. The verdict should remain conditional, but the revision requirements must include a corrected or clearly normalized sustainability computation and a sensitivity analysis of the weights, not merely the latter. I therefore leave the reader's conditional verdict unchanged while flagging the more specific technical defect.","tokens_in":18395,"tokens_out":6123,"duration_ms":54919,"concrete_test":"Recompute STr and SInf for the three settings from Eqs. (6)-(8) using the Table IV inputs under several plausible normalizations: (i) NRMSE as EVal, (ii) EVal expressed as a percentage, and (iii) DS in kB, MB, or GB. If no single normalization reproduces all six STr/SInf entries, Table IV is internally inconsistent and the S ranking is unreliable. Additionally, vary the weights over a grid (e.g., α,β,γ in {0.2, 0.33, 0.5, 0.6} summing to 1, and α',β' in {0.3, 0.5, 0.7}) to test whether Federated remains the minimal-S setting under any reasonable weighting.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim requires Table IV's S=30.24 for Federated versus 71.68 for Individual and 132.03 for Centralized. But plugging the table's own inputs into Eqs. (6)-(8) with the stated weights does not reproduce the reported STr or SInf values. For Individual, (1+1.92)^(1/3)*(1+13.03)^(1/3)*(1+0)^(1/3) = 3.45, not 40.98. Matching 40.98 requires dropping all exponents, but then Federated would give (2.38)*(15.06)*(218) = 7810, not 19.27. Similarly, SInf for Individual would be sqrt(1.92)*sqrt(0.048) = 0.303, not 1.75. The table may be using different normalizations or error metrics, but none is stated or derivable from the text. Since the headline sustainability advantage is computed from these unreproducible numbers, the conclusion that FL achieves the best accuracy-energy-communication trade-off is not supported by the evidence as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a case study of federated learning (FL) for cellular traffic forecasting on real LTE data from seven Barcelona base-station sites. It compares three training settings—Individual, Centralized, and Federated—using an LSTM model, reporting prediction error (NRMSE/MAE) and a sustainability indicator S that combines error, training energy, transmitted data size, and inference energy. The paper also reports ablations on outlier handling, model aggregation, client selection, personalized fine-tuning, and exogenous data enrichment. The central claims are that the federated setting achieves the lowest average prediction error (NRMSE 1.385) and the best accuracy-energy-communication trade-off (S = 30.24), making FL a privacy-preserving and sustainable alternative for traffic forecasting.","tokens_in":18598,"tokens_out":5995,"duration_ms":53039,"significance":"If the claims held, this would be a useful empirical benchmark for FL in mobile traffic forecasting, with the strengths of using a real operator dataset, running 10 seeds, ensuring equal dataset accesses across settings, and providing source code. The ablation studies cover relevant practical aspects (outliers, aggregation, client selection, personalization, exogenous data) that are often missing in FL case studies. However, the main sustainability conclusion rests on a non-reproducible computation in Table IV, and the headline accuracy advantage over centralized learning is small and not supported by any statistical test. The case-study nature and the hand-chosen sustainability weights further limit the generality of the conclusions.","major_comments":[{"comment":"The reported STr, SInf, and S values in Table IV cannot be reproduced from the paper's own formulas and the input values given in the same table. Using alpha=beta=gamma=1/3 and alpha'=beta'=0.5, the Individual setting yields STr = (1+1.92)^(1/3)*(1+13.03)^(1/3)*(1+0)^(1/3) ≈ 3.45 and SInf = sqrt(1.92)*sqrt(0.048) ≈ 0.30, giving S ≈ 1.05, not the reported STr=40.98, SInf=1.75, S=71.68. For the Federated setting, the same formulas give STr ≈ 19.84 and SInf ≈ 0.20, not 19.27 and 1.57. No alternative normalization, error metric, or rounding convention is stated. Since the text uses Table IV to state that 'FL's superiority over other settings is proven', the sustainability claim is unsupported as written. Please correct the table, state the exact definitions of EVal and ETest and any normalization used, or provide the script that computes S.","section":"Section III-B, Eqs. (6)-(8) and Table IV"},{"comment":"The headline accuracy result is Federated NRMSE 1.385 ± 0.044 versus Centralized 1.434 ± 0.003 and Individual 1.92 ± 0.0235. The difference between Federated and Centralized is only 0.049, which is smaller than the reported standard deviation of the Federated mean, and no significance test (e.g., paired t-test over seeds or per-site Wilcoxon test) is reported. The sentence that the federated setting 'achieves the lowest average error of 1.385' therefore overstates the evidence; a confidence interval or significance test is needed to support the claimed superiority.","section":"Section V-A, Table III"},{"comment":"The sustainability indicator S is a weighted product with hand-chosen exponents alpha=beta=gamma=1/3 and alpha'=beta'=0.5. The paper provides no sensitivity analysis for these weights. Because Table IV shows that the Individual setting consumes less training energy (13.03 Wh) than Federated (14.06 Wh), a weight profile that favors energy over accuracy could change the ranking among settings. The claim that 'FL gets the lowest S value ... thus achieving the best trade-off' needs robustness checks; please report how S changes under at least a few alternative weight choices.","section":"Section III-B and Table II"}],"minor_comments":[{"comment":"The introduction says the data were collected at 'five base stations', but the dataset section lists seven sites/splits (LCCN, PS, EB, S1, S2, E1, E2); please align the descriptions.","section":"Introduction and Section IV-A"},{"comment":"The variables EVal and ETest in Eqs. (7) and (8) are not explicitly mapped to a specific error metric (NRMSE or MAE); please define them precisely in the text.","section":"Section III-B"},{"comment":"The total training energy is reported as 13 Wh in Table III and 13.03 Wh in Table IV, and as 14 Wh versus 14.06 Wh for Federated; use consistent significant figures across tables.","section":"Tables III and IV"},{"comment":"The first contribution bullet says 'we introduce ... a novel indicator', but Section III-B says the indicator S was 'introduced in [28]'; please rephrase to 'adapted' or 'applied' to avoid inconsistency.","section":"Section I, contributions"},{"comment":"Reference [48] is cited as 'Tzu et al.' in the aggregation subsection; the actual first author is Hsu et al., so the citation text should be corrected.","section":"Section V-B"}],"recommendation":"major_revision","confidential_remarks":"The non-reproducibility of Table IV is the most serious issue: the central 'FL is most sustainable' claim is not supported by the evidence as currently written. The authors should be asked to provide the exact computation or the code, and to correct the table or the formulas. The accuracy advantage over centralized learning is also marginal and not statistically tested, so the paper's claims should be moderated. If the corrected sustainability numbers change the ranking, the main conclusion of the paper may need to be revised; this is fixable within the scope of a major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid applied case study on federated learning for mobile traffic forecasting, with a new real-world dataset and a broad set of ablations. The headline result — that FL gives the best accuracy-energy trade-off — is undercut by an internal inconsistency in the sustainability metric, and the accuracy advantage over centralized training is small and untested. That said, the empirical work is honest and the paper deserves a serious referee if the authors can reconcile the numbers.\n\nWhat's actually new: the Barcelona PDCCH data, the comparison of nine aggregation strategies, ablation of outlier detection methods, client-selection sensitivity, local fine-tuning, and exogenous feature integration via Upgini. The experimental protocol is reasonably careful: 10 seeds, equal dataset accesses, and the code is linked. Credit where due — the paper doesn't oversell the mixed results of exogenous features, and the client-exclusion analysis is a nice touch.\n\nThe soft spots are real. First and most important: Table IV does not reproduce from Eqs. (6)–(8). For the Individual row, plugging in NRMSE 1.92, training energy 13.03, and zero transmitted data gives STr ≈ 3.45, not 40.98. To get 40.98 you'd have to drop the exponents entirely, but then the Federated row balloons to ~7800, not 19.27. SInf has the same problem. So the paper's central claim that FL is the most sustainable setting is not supported by the evidence as written — this is not just a sensitivity-to-weights concern, the stated formula and the table disagree. Second, the accuracy difference between federated (1.385) and centralized (1.434) is tiny and no significance test is reported; the text calling FL's superiority \"proven\" overreaches. Third, LSTM layer counts and hidden sizes are missing, and the dataset itself is not released, so full reproducibility is limited.\n\nWho's this for? Applied ML researchers in telecoms who want a navigable map of FL design choices on a real traffic-forecasting task. Not a breakthrough, but the ablations are worth having.\n\nRecommendation: send it to peer review, but with a clear request to fix the sustainability table — either correct the equations or the reported S values — and to add significance testing for the main comparison. If the S numbers can't be reproduced, the sustainability conclusion should be downgraded or removed.","headline":"Useful applied FL study with a new Barcelona dataset and solid ablations, but the sustainability table doesn't reproduce from the paper's own equations — fix that before trusting the main claim.","tokens_in":19190,"tokens_out":2885,"would_cite":false,"duration_ms":28358,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Federated learning beats local and centralized training for mobile traffic forecasting on real base-station data, the paper claims, and does so while saving communication and preserving privacy.","keywords":["federated learning","mobile traffic forecasting","long short-term memory (LSTM)","sustainability indicator","non-IID data","client selection","exogenous data enrichment","5G/6G networks"],"falsifier":"Recompute the sustainability score S from the paper's own Table IV numbers with the energy exponent raised from 1/3 to, say, 0.6; if individual training's lower energy (13.03 Wh versus 14 Wh) then beats federated learning's score, the conclusion that FL is the most sustainable setting fails.","tokens_in":18198,"feed_emoji":"📡","tokens_out":8663,"duration_ms":76594,"temperature":0.7,"pith_summary":"This paper argues that federated learning—where base stations exchange model updates rather than raw measurements—is the best among three training regimes for cellular traffic forecasting on a real seven-site dataset from Barcelona. The authors compare a federated LSTM model with each base station training alone and with all data pooled at one central server, controlling for the number of dataset accesses. Federated learning achieves the lowest average prediction error (NRMSE 1.385) and the best composite sustainability score (S = 30.24), which balances error, training energy, transmitted data size, and inference energy. The result matters because mobile traffic forecasts feed resource allocation and anomaly detection in 5G and future 6G networks, where a privacy-preserving option that does not sacrifice accuracy or energy efficiency is directly useful to operators.","feed_headline":"Federated learning wins traffic forecasting on accuracy and energy","feed_subtitle":"On real Barcelona base-station data, FL beats local and centralized training on both error and sustainability.","key_machinery":"The load-bearing mechanism is the federated averaging loop around an LSTM predictor: selected base stations download the global model, train locally for a few epochs, send back weight updates, and the server combines them as a dataset-size-weighted average. The paper's main quantitative instrument is the composite sustainability indicator $S = S_{\\mathrm{Tr}} \\times S_{\\mathrm{Inf}}$, with $S_{\\mathrm{Tr}} = (1 + E_{\\mathrm{Val}})^\\alpha(1 + C_{\\mathrm{Tr}})^\\beta(1 + D_S)^\\gamma$ and $S_{\\mathrm{Inf}} = E_{\\mathrm{Test}}^{\\alpha'} C_{\\mathrm{Inf}}^{\\beta'}$, where the hand-set weights are $\\alpha = \\beta = \\gamma = 1/3$ and $\\alpha' = \\beta' = 0.5$; the tiny transmitted model size is what gives FL its large advantage. The LSTM itself (stacked recurrent layers followed by a ReLU feed-forward head and a linear projection) is adopted because an earlier study found it the best accuracy-to-efficiency balance among RNN, CNN, GRU, and transformer alternatives.","core_discovery":"The paper's central claim is that on the Barcelona PDCCH dataset, federated collaboration dominates both alternatives: it reaches an average NRMSE of 1.385, below centralized training's 1.434 and individual training's 1.92, while consuming about 14 Wh of training energy, roughly 8% more than individual training and 10% less than centralized training. On the sustainability indicator S, federated learning scores 30.24 against 71.67 for individual training and 132.03 for centralized training, because it transmits only 217 kB of model updates per round instead of the full 16,531 kB dataset. Ablation studies reinforce the central claim: FedAvg matches more complex aggregators, isolation-forest capping of outliers improves accuracy, local fine-tuning lowers the federated error by about 10.9%, and enriched exogenous features reduce the average error by 6.55% though with mixed per-site results.","pith_inferences":["If the sustainability weights were shifted toward energy consumption, individual training's lower measured cost (13.03 Wh) could overtake FL's 14 Wh and break the claimed ranking; the missing sensitivity analysis is the natural next check.","Centralized error decays more slowly at long horizons (from step 8 onward), suggesting that pooled data is most valuable for distant forecasts; a hybrid that federates short horizons and centralizes long ones is a testable extension.","Removing site E2 hurt site S1 more than removing S1 itself, and the paper links this to low KL divergence between those datasets; this turns the diagnostic into a concrete candidate rule for data-similarity-aware client selection.","The mixed effect of exogenous enrichment implies feature selection should be per-site rather than global, since sites with one-off events did not benefit from the automatically added features."],"forward_implications":["Operators can obtain near-centralized accuracy without moving raw traffic data off base stations, reducing privacy exposure and data-transfer cost.","Under the paper's matched-access protocol, the privacy-preserving option costs only about 8% more training energy than per-site training, so privacy does not carry a large energy penalty.","Cutting participation from 7/7 to 5/7 sites per round degrades average error by only 4–5%, so client selection is a practical lever for lowering per-round cost.","Local fine-tuning after the federated round improves per-site error by roughly 10.9%, making FL workable across non-IID site traffic.","Exogenous data (holidays, events, calendar features) lowers average federated error by 6.55%, but gains are uneven across sites with anomalous events."],"supporting_citations":[{"why":"Defines FedAvg, the aggregation rule used for the main federated results.","marker":"[11]"},{"why":"Earlier study that established the LSTM choice and the federated traffic-forecasting setup extended here.","marker":"[13]"},{"why":"Supplies the Barcelona PDCCH measurements from the five base-station locations.","marker":"[14]"},{"why":"Introduces the sustainability indicator S that the paper uses to rank the three settings.","marker":"[28]"},{"why":"Gives the LSTM cell equations used in the forecasting network.","marker":"[40]"},{"why":"Provides the power and CO2 measurement library behind the energy figures.","marker":"[42]"},{"why":"Isolation forest, the outlier detection method that improves federated accuracy in the ablation.","marker":"[43]"},{"why":"FedNova, one of the aggregation baselines compared in the model-aggregation study.","marker":"[47]"},{"why":"FedAvgM, another aggregation baseline compared in the model-aggregation study.","marker":"[48]"},{"why":"Automated feature-enrichment library that supplies the exogenous data tested at the end.","marker":"[51]"}],"fun_headline_variants":["Federated learning beats centralized and local for mobile traffic forecasting","FL outperforms on accuracy and energy in Barcelona base-station study","Federated learning: accurate and sustainable for mobile traffic prediction","In mobile networks, federated learning wins on error and energy use","Federated learning dominates traffic forecasting on accuracy and eco-friendliness"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that federated learning is the most sustainable setting rests on the paper's hand-picked weights in the sustainability formula, and those weights are never varied, even though individual training actually uses less energy.","fun_headline_variants_meta":{"raw":{"variants":["Federated learning beats centralized and local for mobile traffic forecasting","FL outperforms on accuracy and energy in Barcelona base-station study","Federated learning: accurate and sustainable for mobile traffic prediction","In mobile networks, federated learning wins on error and energy use","Federated learning dominates traffic forecasting on accuracy and eco-friendliness"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000324,"raw_usage":{"total_tokens":1811,"prompt_tokens":930,"completion_tokens":881,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":791}},"tokens_in":546,"tokens_out":881,"duration_ms":7736,"temperature":1.0,"reasoning_tokens":791,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:46:17.995479+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the sustainability score S from the paper's own Table IV numbers with the energy exponent raised from 1/3 to, say, 0.6; if individual training's lower energy (13.03 Wh versus 14 Wh) then beats federated learning's score, the conclusion that FL is the most sustainable setting fails.","supporting_citations":[{"cited_title":"Federated learning for 5G base station traffic forecasting,","cited_arxiv_id":null,"evidence_quote":"Earlier study that established the LSTM choice and the federated traffic-forecasting setup extended here."},{"cited_title":"Mobile traffic classification through physical control channel fingerprinting: a deep learning approach,","cited_arxiv_id":null,"evidence_quote":"Supplies the Barcelona PDCCH measurements from the five base-station locations."},{"cited_title":"Towards energy-aware federated traffic prediction for cellular networks,","cited_arxiv_id":null,"evidence_quote":"Introduces the sustainability indicator S that the paper uses to rank the three settings."},{"cited_title":"Long short-term memory,","cited_arxiv_id":null,"evidence_quote":"Gives the LSTM cell equations used in the forecasting network."},{"cited_title":"Tackling the objective inconsistency problem in het- erogeneous federated optimization,","cited_arxiv_id":null,"evidence_quote":"FedNova, one of the aggregation baselines compared in the model-aggregation study."},{"cited_title":"Federated visual classification with real-world data distribution,","cited_arxiv_id":null,"evidence_quote":"FedAvgM, another aggregation baseline compared in the model-aggregation study."},{"cited_title":"Upgini - automated data search & enrichment library,","cited_arxiv_id":null,"evidence_quote":"Automated feature-enrichment library that supplies the exogenous data tested at the end."}],"review_version":1}