{"id":"e8b9a48b-25fc-4d24-8a8d-b894905c7123","arxiv_id":"2507.11471","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"On synthetic and Ausgrid energy data, federated LSTM forecasting is less accurate than centralized training for non-linear distributions, and detrending choice matters; differencing helps most in synthetic experiments, while mean or quadratic removal help slightly on real data.","lead":"This paper compares centralized and federated LSTM forecasting on time-series data drawn from extreme-value and log-normal distributions, with and without five detrending methods. It finds that federated models usually lag centralized ones, and that detrending choice matters, though the best method varies by distribution and dataset.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Detrending statistics are never stated to be fit on training data only; if full-series mean/trend are used, the reported detrending gains in Tables V and VII may be lookahead artifacts, not genuine forecasting improvements.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the paper never states whether detrending statistics are estimated on the training portion only or on the full series before the chronological split. I agree that this is the most important unresolved condition for the paper's central claim. The concern is not speculative: Eq. 5 globally subtracts a full-series mean, and Eqs. 6 and 7 fit global regression curves, so any implementation that applies these equations before the 90/10 split necessarily uses validation points in the transform. The reported conclusions about detrending effectiveness, especially the per-distribution ranking of methods, are therefore not trustworthy without clarification or a causal reimplementation. I do not escalate the verdict beyond the reader's CONDITIONAL because the issue is fixable: the authors can disclose the fitting procedure or rerun the experiments with training-only statistics, and the rest of the paper's observations about FL versus centralized training on vanilla data are less affected by this particular leakage. The absence of a separate test set and error bars are additional concerns, but the detrending-before-split ambiguity is the one that most directly threatens the second headline claim. The proposed test is concrete and would settle whether the reported gains survive under leakage-free preprocessing.","tokens_in":16410,"tokens_out":3286,"duration_ms":40877,"concrete_test":"Re-run all synthetic and Ausgrid detrending experiments with strictly causal preprocessing: estimate mean, linear, and quadratic detrending coefficients (and any moving-average statistics) using only the training prefix, i.e., the first 90% of each client's chronologically sorted series, then apply those fixed statistics to transform the validation 10%. Compare the resulting MSE, RMSE, and MAE in Tables V and VII against the reported values. If the validation losses for mean/linear/quadratic detrending change materially (e.g., by more than 5% relative) or if the best detrending method per distribution changes, the reported detrending gains are partly leakage artifacts rather than genuine forecasting improvements.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The weakest load-bearing condition is the interaction between detrending and the 90/10 chronological split. Section IV-A states that each client's data is chronologically sorted and the first 90% is used for training with the last 10% as validation, but Sections IV-B and V do not state whether the detrending statistics are estimated on the training prefix only or on the full client series before the split. Equations 5, 6, and 7 require global statistics: a full-series mean in Eq. 5, and full-series linear or quadratic regression coefficients in Eqs. 6 and 7. If these are computed over all 10,000 points, the validation window contributes to the transform applied to both training inputs and validation targets, so the detrended validation values contain information from the future relative to the forecasting task. The central claim that 'appropriate detrending techniques improve FL performance, reducing loss across different data distributions' is then not a clean statement about forecasting: the apparent improvements in Tables V and VII for mean/linear/quadratic detrending could be an artifact of the detrending transform having already seen the validation segment. Differencing (Eq. 3) is causal, and the moving-average form in Eq. 4 uses only current and past lags as written, so the leakage risk is concentrated in exactly the methods for which the paper does not report fitting details. This concern is load-bearing because it targets the paper's second headline contribution, not a peripheral metric.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies how non-linear, non-stationary time-series distributions (generalized extreme value, log-normal, and a mix) affect LSTM forecasting under FedAvg versus centralized training, and evaluates five detrending techniques. Experiments on synthetic data and Ausgrid substation data report validation MSE, RMSE, and MAE; the central claims are that FL underperforms centralized training on such data and that choosing an appropriate detrending method reduces forecasting loss across distributions.","tokens_in":16693,"tokens_out":6273,"duration_ms":68945,"significance":"If confirmed, the comparison provides useful empirical evidence for practitioners choosing between FL and centralized training for non-linear IoT time-series forecasting, and it highlights that the detrending choice matters in FL. The use of controlled synthetic data with known distribution families is a strength, as is the inclusion of a real-world energy dataset and the systematic comparison of six preprocessing settings. The paper does not introduce theory or release code, so its contribution is primarily empirical; however, the current experimental reporting lacks repeated runs, a held-out test set, and precise preprocessing details, which limits the strength of the stated claims.","major_comments":[{"comment":"The manuscript never states whether detrending statistics are estimated on each client's training prefix or on the full series before the 90/10 chronological split. Equations (5)–(7) define the mean, linear, and quadratic coefficients over the whole series, and Section IV-A only describes the split for training and validation data. If the full series is used, the validation segment contributes to the transform, so the detrending gains in Tables V and VII could be lookahead artifacts rather than genuine forecasting improvements. Please clarify the fitting window; if the full series was used, re-run the experiments with the statistics computed on the training portion only.","section":"§IV-A and §II-C (Eqs. 5–7)"},{"comment":"All results are reported as single point estimates with no indication of random seeds, number of runs, or error bars. For example, in Table V the FL gen-extreme MSE for differencing is 0.00554 versus 0.00677 for moving average; without variance information it is impossible to tell whether this difference is meaningful. Because the paper's second contribution is a recommendation to select an \"appropriate\" detrending method from exactly these numbers, the absence of repeated runs is load-bearing. Please report means and standard deviations over multiple seeds and, where possible, a significance test.","section":"Tables IV–VII"},{"comment":"No held-out test set is used: the validation split described in Section IV-A serves both to select the best detrending method and to report final performance. For instance, the claim that quadratic detrending \"yielded the best results\" on Ausgrid gen-extreme data is based on the same validation losses in Table VII that were used for method selection. This double use can overstate the benefit of the chosen method. Please add a held-out test segment after model selection and report performance on that segment.","section":"§VI.B.2 and Tables VI–VII"},{"comment":"The centralized RMSE values are internally inconsistent with the reported MSE values for gen-extreme data. For the moving-average row, MSE = 0.0000011 but RMSE = 0.0077, whereas the square root of the MSE is approximately 0.00105; similar mismatches appear for the linear (MSE = 0.0000068, RMSE = 0.0072) and quadratic (MSE = 0.0000098, RMSE = 0.0074) rows. Since Table VII is the basis for the real-world detrending conclusions, these inconsistencies must be corrected or the affected conclusions revisited.","section":"Table VII"},{"comment":"Several parameters required to reproduce the experiments are not reported. Equation (4) defines a moving average with a window of size p, but no value of p is given anywhere; Section IV-B describes the synthetic data generation only qualitatively (\"a sine function\", \"an additional offset\") without the exact formulas or parameter values; and Section IV-D mentions a grid search over lookback and lookahead but not the search ranges. These omissions are material because the moving-average and synthetic-data results are central to the paper's conclusions.","section":"§II-C and §IV-B"}],"minor_comments":[{"comment":"The denominator in the moving-average formula is written as n but the sum runs over p terms; the notation should be made consistent, with p defined.","section":"Eq. (4)"},{"comment":"The fourth panel in the log-normal figure is labeled \"Subtracting mean\" twice; the second occurrence should be \"Linear model\" to match the other figures.","section":"Figure 9"},{"comment":"The experiment count is stated inconsistently: Section V says eighteen experiments on each dataset, while Section VI.A.2 says 15 detrending experiments and 30 total experiments per setup; please clarify the counting.","section":"Section V and Section VI.A.2"},{"comment":"The paper alternates between \"log norm\" and \"log normal\"; standardize the terminology, especially in the abstract and tables.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"My primary concern is the potential lookahead leakage in the detrending preprocessing. If the authors confirm that statistics were computed on the training portion only and correct the experimental reporting, the paper could be salvageable; in the current form I cannot support acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a legitimate comparison study — five detrending methods across GEV, log-normal, and mixed clients under FedAvg, on synthetic and Ausgrid data. The matrix itself is new, and practitioners would care about the answer. The paper is clearly written and the synthetic generation is more careful than most.\n\nThe problem is that the second headline claim, that 'appropriate detrending improves FL performance,' rests on an unstated preprocessing detail. The paper never says whether the detrending statistics (mean, linear/quadratic coefficients) are estimated on the 90% training prefix or on the full series before the 90/10 split. Equations 5–7 are written with global statistics. If the full series is used, the validation segment leaks into the detrending transform, and the gains in Tables V and VII for mean/linear/quadratic detrending become lookahead artifacts, not forecasting improvements. Differencing is causal, and moving-average as written uses past lags, so the suspicion falls on exactly the methods where the paper reports the most interesting results. This is not a peripheral nit; it is the core of contribution (2).\n\nOther soft spots, in decreasing severity. No repeated runs or error bars anywhere; differences like MSE 0.00282 vs 0.00286 are noise. The validation set is used both to pick the 'appropriate' detrending method and to report the final loss, which inflates the apparent benefit. On the real-world Ausgrid data, most detrending methods are flat or worse than baseline; the 'improvement' is a few basis points. The paper also omits practical details: Ausgrid is 15-min data but the model section only specifies hourly sampling, the substation distribution assignment is not described, and no normalization is mentioned. The FL-vs-centralized conclusion is fine, but that is not a new result.\n\nWhat is genuinely useful: the comparison matrix, and the observation that the best detrending method differs by distribution and by training paradigm. That is a good starting point for a more rigorous study.\n\nBottom line: worth a serious referee and a major-revision decision. The authors need to state clearly that detrending was fit on training data only, or re-run the experiments that way, and add error bars. If the leakage concern lands, the detrending contribution collapses to differencing-only; if it does not, this becomes a solid empirical benchmark. I would bring it to reading group, mainly as a cautionary example.","headline":"A useful but under-specified benchmark; the detrending claims hinge on an unstated fit-before-split choice that risks lookahead leakage.","tokens_in":17228,"tokens_out":5504,"would_cite":false,"duration_ms":60879,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that federated learning with FedAvg is less accurate than centralized training on non-linear time series, and that the right detrending method closes much of that gap.","keywords":["federated learning","time-series forecasting","non-linear data distributions","detrending","generalized extreme value distribution","log-normal distribution","LSTM","non-IID data"],"falsifier":"Re-run the synthetic and real-world experiments with all detrending statistics computed inside the training window only, using a walk-forward or expanding-window estimation, and compare the validation MSE values against Tables IV-VII; if the detrending advantage largely disappears, the reported gains were lookahead artifacts.","tokens_in":16200,"feed_emoji":"📉","tokens_out":7601,"duration_ms":75149,"temperature":0.7,"pith_summary":"The paper claims that federated learning (FL) with FedAvg produces worse forecasts than centralized training when clients hold non-linear time series drawn from generalized extreme value or log-normal distributions, and that the gap shrinks when the series are detrended before training. It reports experiments with ten clients, an LSTM forecaster, five detrending techniques (differencing, moving average, mean subtraction, linear regression, quadratic regression), and both synthetic and real-world (Ausgrid network) data. The practical message is that detrending is not a neutral pre-processing step: the technique that works best depends on the client data distribution, and the wrong choice can increase validation loss. A caveat is that the paper never states whether detrending statistics are estimated on the training segment only, so the reported gains could partly come from lookahead.","feed_headline":"Federated learning trails centralized on non-linear time series","feed_subtitle":"The right detrending method cuts forecast loss, but the best choice depends on the data distribution.","key_machinery":"The carrying protocol is a ten-client comparison between centralized training and FedAvg, using a one-layer LSTM (24-step input, 2-step output) as the shared forecaster. Each client holds a univariate series drawn from either the generalized extreme value (GEV) or log-normal family, with parameters chosen so values fall roughly in a 2-to-20 range. The manipulation is detrending, applied before the chronological 90/10 train/validation split, using five standard techniques: first differencing, sliding-window moving average removal, mean removal, linear regression subtraction, and quadratic regression subtraction. These methods convert each non-stationary series into a near-stationary residual series; the paper's finding is that the optimal conversion depends on the underlying distribution.","core_discovery":"On its own terms, the paper's central discovery is that non-linear, non-stationary client data distributions degrade the performance of a standard FedAvg-LSTM forecaster, and that an appropriate detrending step recovers most of the loss. Without detrending, FL validation MSE on synthetic gen-extreme data was 0.00844 versus 0.00273 for centralized training, and the log-normal case rose from 0.00589 to 0.01583; mixing both distributions widened the FL gap further. With detrending, FL MSE fell to 0.00554 for gen-extreme (differencing) and 0.01339 for log-normal (differencing) on synthetic data, while on the real-world Ausgrid data the lowest FL losses came from quadratic detrending for gen-extreme and mean subtraction for log-normal. The paper reads these results as evidence that the choice of detrending method is distribution-dependent and that detrending should be part of the FL pipeline.","pith_inferences":["If the detrending gains survive a strictly causal re-run, a natural extension is per-client detrending choice communicated as metadata to the server, which could beat any single global method.","The disagreement between the synthetic result (differencing best) and the real-world result (quadratic or mean removal best) suggests the optimal method depends on trend shape and noise, not just the marginal distribution; a testable hypothesis is that differencing wins for stochastic trends and polynomial removal for smooth deterministic trends.","A mechanism the authors leave implicit is that detrending makes clients' residual series more alike, reducing FedAvg gradient conflict; measuring client-weight divergence before and after detrending would test this directly.","Because the paper does not specify when detrending statistics are computed, its FL-versus-centralized comparison should be re-run under strictly causal preprocessing before the specific loss numbers are used for design decisions."],"forward_implications":["FL practitioners should expect a non-linear or heavy-tailed client distribution to widen the gap between federated and centralized forecast error, independent of communication constraints.","Detrending is not neutral in FL: the wrong method can raise validation loss, so the choice should be matched to the client data distribution.","On synthetic series, first-order differencing gave the lowest federated validation error for both GEV and log-normal clients, suggesting residual-based training helps the global model.","The mixed-distribution experiment implies that the presence of log-normal clients disproportionately raises the global FedAvg validation loss, which may inform client selection."],"supporting_citations":[{"why":"Cited as the FedAvg aggregator used in all federated experiments; the paper's FL results depend on this aggregation rule.","marker":"[10]"},{"why":"Supports the premise that federated learning is less effective on non-iid data, motivating the study.","marker":"[13]"},{"why":"Provides the prior result that neural network forecasters perform better after trend and seasonality are removed, motivating the detrending comparison.","marker":"[27]"},{"why":"Defines the generalized extreme value distribution used to generate the gen-extreme client data.","marker":"[30]"},{"why":"Defines the log-normal distribution used to generate the log-norm client data.","marker":"[31]"},{"why":"The Flotilla framework used to run the federated learning experiments on Docker containers.","marker":"[35]"},{"why":"Supplies the real-world Ausgrid distribution zone substation dataset used to validate the results.","marker":"[36]"}],"fun_headline_variants":["Detrending boosts federated learning on non-linear time series","FedAvg stumbles on non-linear data, detrending fixes it","Detrending choice matters for federated non-linear forecasting","Federated learning lags on skewed time series without preprocessing","Distribution-aware detrending improves federated forecast accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The experiments assume that detrending statistics (moving average, mean, and regression coefficients) are computed without looking at the validation portion of each time series, but the paper never states that they are estimated on the training portion only.","fun_headline_variants_meta":{"raw":{"variants":["Detrending boosts federated learning on non-linear time series","FedAvg stumbles on non-linear data, detrending fixes it","Detrending choice matters for federated non-linear forecasting","Federated learning lags on skewed time series without preprocessing","Distribution-aware detrending improves federated forecast accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000225,"raw_usage":{"total_tokens":1518,"prompt_tokens":1056,"completion_tokens":462,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":672,"completion_tokens_details":{"reasoning_tokens":377}},"tokens_in":672,"tokens_out":462,"duration_ms":5095,"temperature":1.0,"reasoning_tokens":377,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:08:01.104276+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the synthetic and real-world experiments with all detrending statistics computed inside the training window only, using a walk-forward or expanding-window estimation, and compare the validation MSE values against Tables IV-VII; if the detrending advantage largely disappears, the reported gains were lookahead artifacts.","supporting_citations":[{"cited_title":"Fast-convergent federated learning with adaptive weighting.IEEE Transactions on Cognitive Communications and Networking, 7(4):1078–1088, 2021","cited_arxiv_id":null,"evidence_quote":"Supports the premise that federated learning is less effective on non-iid data, motivating the study."},{"cited_title":"Neural network forecasting for seasonal and trend time series.European journal of operational research, 160(2):501– 514, 2005","cited_arxiv_id":null,"evidence_quote":"Provides the prior result that neural network forecasters perform better after trend and seasonality are removed, motivating the detrending comparison."},{"cited_title":"Modeling extreme climatic events using the generalized extreme value (gev) distribution","cited_arxiv_id":null,"evidence_quote":"Defines the generalized extreme value distribution used to generate the gen-extreme client data."},{"cited_title":"Financial data analysis with two symmetric distri- butions.Astin Bulletin, 31(1):187–211, 2001","cited_arxiv_id":null,"evidence_quote":"Defines the log-normal distribution used to generate the log-norm client data."},{"cited_title":"Towards a modular federated learning framework on edge devices","cited_arxiv_id":null,"evidence_quote":"The Flotilla framework used to run the federated learning experiments on Docker containers."},{"cited_title":"https://www.ausgrid.com.au/ Industry/Our-Research/Data-to-share/Distribution-zone-substation-data,","cited_arxiv_id":null,"evidence_quote":"Supplies the real-world Ausgrid distribution zone substation dataset used to validate the results."}],"review_version":1}