{"id":"8a0e9fc8-130d-48c5-9e1b-2ecc70071980","arxiv_id":"2508.16641","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"On one Belgian electricity load series, bagging, regression stacking, and residual correction reduce MSE relative to standalone Lag-Llama and AutoGluon forecasts, though the reported numbers are inconsistent and lack error bars.","lead":"This paper tests standard ensemble tricks (bagging, regression stacking, residual correction) on top of pretrained time-series forecasting models, using one Belgian electricity load dataset. The main finding is that mixing a foundation model with AutoGluon forecasts lowers error metrics and gives calibrated-looking prediction intervals, but the evidence is thin and inconsistent.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 67% residual-correction gain in Table 3 appears to be an in-sample artifact: Algorithm 3 trains AutoGluon on the error series without a stated train/test split, and the Lag-Llama baseline varies across tables, so the headline consistent-improvement claim is not yet reliably supported.","rationale":"I read the paper as an empirical case study whose central contribution is the claim that statistical post-processing (bagging, regression stacking, residual correction, PI construction) consistently improves TSFM forecasts on the Belgium load dataset. The strongest supporting evidence is Tables 1-3 and Figures 1-9. The residual modeling in Section 3.4 and Algorithm 3 is the weakest link because the algorithm computes errors against observed values and then trains a second model on those errors without specifying the split that would make the residual predictions out-of-sample. This is the same concern the reader flagged; I agree with that assessment. I also notice a secondary issue: the Lag-Llama baseline MSE differs across tables for what look like the same settings (1-week: 1690/1288/1413; 3-week: 1722/816/180). If these are different evaluation subsets or different definitions of the point forecast, the paper should say so explicitly; if not, the comparisons in the abstract are not well-defined. I am not accusing the authors of misreporting; the public code allows a direct check. Given that the residual claim is only one of four bullets, I do not think this changes the reader's overall CONDITIONAL verdict, but it reinforces the need for a reproducible walk-forward experiment and reconciled tables before the headline claim is accepted.","tokens_in":9367,"tokens_out":11251,"duration_ms":118179,"concrete_test":"Use the public GitHub repository to rerun Algorithm 3 with a strict walk-forward split: fit AutoGluon on residual errors only from the first 70% of the series (the Section 4 training set), then produce one-hour-ahead residual predictions for the held-out 20%; recompute the corrected MSE. Also rerun the Lag-Llama one-step baseline on that same held-out period and check whether it equals Table 3's 180 (3-week) and 1413 (1-week) entries. If the 180-to-59 gain requires access to test-period residuals, or if the baseline value does not reproduce, the residual-correction claim is an artifact and the central consistent-improvement statement needs revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Algorithm 3's residual-correction step is the least constrained part of the empirical chain. Step a computes e_t = yhat_t - y_t for every point; Step b says 'Train a statistical time series model (AutoGluon) using error series e_t' with no chronological split. If the e_t series used to fit AutoGluon includes the evaluation period, the residual forecast is an in-sample fit, and the Table 3 improvement from 180 to 59 (67% at 3-week context) is not a genuine forecast gain. The only operational detail given, that the same context lengths are used to predict the next one-hour error, does not by itself prevent leakage. This is load-bearing because this 67% figure is the sole quantitative evidence for the residual-modeling bullet in the abstract. It is compounded by the fact that the Lag-Llama baseline itself is not stable across tables: for the 3-week context it is 1722 in Table 1, 816 in Table 2, and 180 in Table 3 (1-week: 1690, 1288, 1413). Unless these are different evaluation windows or prediction summaries, the reported improvements cannot be compared.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whether simple statistical post-processing techniques can improve the point forecasts and uncertainty estimates produced by time-series foundation models. Using the Belgium Electricity Short-Term Load Forecasting dataset, the authors apply bootstrap-based bagging to Lag-Llama one-step-ahead samples, linear regression stacking of Lag-Llama with AutoGluon forecasts, ensemble prediction intervals, residual correction of Lag-Llama errors via AutoGluon, and iterative error feedback with TimeGPT. The central claim is that these hybrids consistently outperform standalone foundation models across 1-24 hour horizons, with regression ensembles giving the lowest MSE, bagging reducing long-context errors, residual modeling correcting bias, and prediction intervals achieving near-nominal coverage. Evidence is presented in Tables 1-3, Figures 1-9, and Appendix B.","tokens_in":9593,"tokens_out":4511,"duration_ms":49086,"significance":"If the claims were reliable, the paper would offer a practical, low-cost recipe for improving the accuracy and calibration of time-series foundation models on operational load data, and the public code would aid reproducibility. The paper's strengths are its clear algorithmic descriptions, use of publicly available data, and the apparent ease of re-running the AutoGluon/Lag-Llama pipeline. However, the main empirical claims are not currently trustworthy because the same baseline setting receives different numbers across tables, the residual-correction step does not specify a chronological train/test split, and the bagging procedure is mathematically indistinguishable from a simple mean of the model's draws under the paper's own definition of the point prediction. These issues are load-bearing for the abstract's headline claims, so the contribution, while plausible, is not yet supported by the reported evidence.","major_comments":[{"comment":"The Lag-Llama baseline is not stable across tables for the same context length and forecast setting. For a 3-week context the Lag-Llama MSE is 1722 in Table 1, 816 in Table 2, and 180 in Table 3; for a 1-week context it is 1690, 1288, and 1413. Tables 2 and 3 both describe one-step-ahead forecasts, so these numbers cannot all refer to the same evaluation window. Because the percentage improvements in the abstract and Section 5 are computed from these inconsistent baselines, the comparisons are not meaningful as reported. Please specify the exact evaluation period and prediction summary for each table and reconcile the numbers.","section":"Tables 1-3"},{"comment":"The residual-correction procedure is missing a train/test split. Step b of Algorithm 3 trains AutoGluon on the error series e_t and then slides one hour, but nowhere is it stated that the training window excludes the evaluation period. If the fitted error series includes the test horizon, the 67% improvement in Table 3 (180 to 59 MSE at 3-week context) is an in-sample artifact rather than a forecast gain. Please provide a strict chronological walk-forward protocol for training the error model and re-report the residual-correction results under that protocol.","section":"Section 3.4, Algorithm 3"},{"comment":"The bagging procedure as written is equivalent to the arithmetic mean of the 100 draws. Drawing 40 values with replacement, averaging, and repeating 100 times yields bootstrap means whose average converges to the sample mean of the original 100 draws. Since Algorithm 3 defines the Lag-Llama point estimate as exactly this mean, the 23-54% improvements in Table 1 require an explanation of how the bagged forecast differs from the point forecast. Please state how Lag-Llama's point prediction is generated (mean, median, or a single draw) and, if it is the mean, explain the mechanism by which bagging can improve over it.","section":"Section 3.1 and Algorithm 1"},{"comment":"All headline improvements are single point estimates on one dataset with no error bars, repeated runs, or significance tests. This would be a secondary concern if the numbers were internally consistent, but combined with the baseline contradictions and the ambiguity in Algorithm 3, the claim of 'consistent' improvements across horizons is not yet supported. Please report means and standard deviations over multiple seeds or a bootstrap of the evaluation period, and state whether differences are statistically reliable.","section":"Section 5, Tables 1-3"}],"minor_comments":[{"comment":"Section 4 describes a 70/10/20 train/validation/test split, while Algorithm 2 states an 80/20 split. Please align these descriptions.","section":"Section 4 vs Algorithm 2"},{"comment":"The title and abstract mention boosting, but no boosting algorithm is defined; residual modeling and iterative error feedback are presented as different methods. Please either remove the term 'boosting' or define a formal boosting step.","section":"Title and Section 5.1"},{"comment":"The prose describing the regression coefficients is internally inconsistent: for the 1-week model the text says AutoGluon coefficients 'progressively drop to 0.613 at step 24' after starting at 0.602, which is an increase, and for the 3-week model it says the coefficients 'peak at step 4 (0.515)' after starting at 0.579, which is a decrease. Please correct these statements.","section":"Section 5, Figures 4-6"},{"comment":"The prediction-interval results are described only as 'near-nominal coverage'; no empirical coverage rates or interval-width values are reported. Please add a table or explicit numbers.","section":"Section 3.3 and Section 5"},{"comment":"One citation in Section 2.3 contains the placeholder '?' instead of a proper reference. Please complete the citation.","section":"Section 2.3 references"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads like a condensed thesis chapter and would benefit from expanded methodological detail. The main concern is not the overall idea but whether the reported improvements are real: the residual-correction pipeline may leak test information, and the bagging claim is mathematically under-specified. If the authors cannot reconcile Table 1 with Table 3 or provide a genuine walk-forward residual experiment, the paper may not be salvageable in its current scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [X],\n\nThis is a short applied case study, not a methods paper. The novelty is the packaging: standard bagging, linear stacking, residual correction, and normal-approximation intervals applied to Lag-Llama and TimeGPT on one Belgian electricity load dataset. Nothing in the toolbox is new, and the title's \"boosting\" is really iterative error feedback. I would not fault the authors for that if the empirical claims held up.\n\nCredit where due: the code is public, the main stacking pipeline uses a clear chronological 80/20 split, the weights and PI widths are reported in detail, and the overall direction—ensembling stabilizes a single foundation model—is consistent with what forecast combination should do. The bagging gains and stacking gains are plausible.\n\nThe soft spots are concentrated in the residual-correction section and in the tables. Algorithm 3 fits AutoGluon to the residual series without saying where training ends. If the fitted error series includes the evaluation period, the headline 67% improvement (180 to 59 MSE at three weeks) is an in-sample artifact. The paper needs to state the split explicitly. On top of that, the Lag-Llama baseline for the same claimed setup changes across tables: 1722 in Table 1, 816 in Table 2, 180 in Table 3. Without reconciliation, the abstract's \"consistently outperform across multiple horizons\" is not supported by the numbers. There are also no error bars or significance tests on any MSE comparison, and the entire evaluation is one dataset, so the generality claims are rhetorical. The limitation note about PI independence is honest, but it does not repair the residual-modeling ambiguity.\n\nFor practitioners, this is a useful starting template. For someone looking for new methodology or theory, there is little here. With the split specified and the tables reconciled, it would be a decent applied case study.\n\nRecommendation: send it to a serious referee. The flaws are fixable and the artifact is reproducible. Ask for a precise residual-modeling split, a reconciliation of Tables 1–3, and at least a bootstrap over evaluation windows for the headline gains.","headline":"A reproducible applied case study of standard ensembling around time-series foundation models, but the residual-correction gain looks like an in-sample artifact and the headline tables don't agree; referee it, require fixes.","tokens_in":10151,"tokens_out":3812,"would_cite":false,"duration_ms":41685,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that frozen transformer-based time-series foundation models can be made materially more accurate, stable, and calibrated by wrapping them in cheap statistical post-processing, with regression ensembles achieving the…","keywords":["time series forecasting","foundation models","Transformers","ensembling","bagging","boosting","uncertainty quantification","electricity load forecasting"],"falsifier":"Run Algorithm 3 with a strict chronological split: fit the AutoGluon error model only on errors strictly before the evaluation window and then score the adjusted forecasts on the held-out period. If the three-week MSE improvement drops far below the reported 67% or turns negative, the residual-correction result is an artifact of look-ahead.","tokens_in":9113,"feed_emoji":"⚡","tokens_out":6602,"duration_ms":63682,"temperature":0.7,"pith_summary":"This paper aims to show that frozen transformer-based time-series foundation models can be made materially more accurate, stable, and calibrated by wrapping them in cheap statistical post-processing instead of fine-tuning. The proposed hybrids, which include bootstrap bagging of the model's predictive draws, linear-regression stacking with an automated statistical forecaster, residual-error correction, and iterative error feedback, are tested on hourly Belgian electricity demand. Across 1- to 24-hour horizons, the paper reports that the hybrids consistently beat standalone foundation models on mean squared error, that regression ensembles are best overall, and that bagging cuts long-context error most. The practical stake is real operational forecasting: better point forecasts and honest prediction intervals from models that are already deployed, without retraining their weights.","feed_headline":"Statistical add-ons cut time-series forecast error by up to 67%","feed_subtitle":"On Belgian load data, bootstrap bagging, regression stacking, and residual correction beat standalone foundation models.","key_machinery":"The load-bearing object is a three-part post-processing pipeline placed on top of a frozen forecaster. First, bagging: Lag-Llama emits $n=100$ one-step-ahead draws; the method samples $b=40$ of them with replacement, averages those draws, repeats this $m=100$ times, and averages again to produce $\\hat{y}^{\\mathrm{bag}}_t$. Second, stacking: weights $(w_1,w_2)$ are fit by linear regression with time-series cross-validation so that $\\hat{y}^{\\mathrm{ens}}_t = w_1 \\hat{y}^{\\mathrm{Lag}}_t + w_2 \\hat{y}^{\\mathrm{AG}}_t$, and the ensemble variance is $\\sigma^2_{\\mathrm{ens}} = w_1^2\\sigma^2_{\\mathrm{Lag}} + w_2^2\\sigma^2_{\\mathrm{AG}}$ with $\\sigma_{\\mathrm{AG}} \\approx (P_{90}-P_{10})/2.5631$; this feeds a 95% interval $\\mu_{\\mathrm{ens}} \\pm 1.96\\sigma_{\\mathrm{ens}}$. Third, residual correction: the error series $e_t=\\hat{y}^{\\mathrm{Lag}}_t - y_t$ is modeled by AutoGluon, and the forecast is adjusted to $\\hat{y}^{\\mathrm{adj}}_t = \\hat{y}^{\\mathrm{Lag}}_t - \\hat{e}_t$; for TimeGPT, residuals are appended as an exogenous channel and forecasting repeats until validation RMSE converges.","core_discovery":"On the Belgium Electricity Short-Term Load Forecasting dataset, the authors find that bootstrap bagging alone lowers Lag-Llama's one-step mean squared error from 1690 to 1297 with a one-week context, from 1722 to 795 with three weeks, and from 914 to 486 with five weeks. A linear regression stack of Lag-Llama and AutoGluon beats both components at every context, reaching MSE 959 versus 1288 and 1074 at one week, 628 versus 816 and 719 at three weeks, and 196 versus 488 and 203 at five weeks. Residual modeling, which fits AutoGluon to the error series, gives a 67% improvement at three weeks (MSE 180 to 59) and 1% at one week, while iterative residual feedback to TimeGPT produces moderate RMSE gains in dynamic settings. Ensemble-based 95% prediction intervals achieve near-nominal coverage and shrink as context length grows.","pith_inferences":["Inference: because bagging only requires a sample of predictive draws, the same bootstrap recipe should transfer to any probabilistic time-series foundation model, not just Lag-Llama; the paper tests it on only one base model.","Inference: the coefficient trends suggest a simple deployment rule, start with AutoGluon-dominated weights at short horizons and shift weight toward the transformer as context and horizon grow, that could be tested online without refitting.","Inference: if a strict chronological check confirms the residual gains, residual modeling doubles as a lightweight online adaptation layer that tracks drifting bias, not just a static post-hoc correction.","Inference: the paper's independence-based interval could be tightened by bootstrap resampling of forecast errors, a calibration fix the authors themselves gesture at in their limitations."],"forward_implications":["If the results hold, practitioners can improve a deployed foundation model's point forecasts and intervals without touching its weights, using only the predictive samples the model already emits.","Longer context windows become more valuable under these hybrids: bagging gains grow with context, and prediction intervals narrow by about 20% when context moves from one to three weeks without losing coverage.","Stacking weights reveal an interpretable division of labor: AutoGluon leads on short contexts and near horizons, while Lag-Llama gains influence as context and forecast horizon grow.","Residual correction targets systematic bias rather than random variance, which is why it delivers the largest gain at the three-week context where bias dominates.","Prediction-interval quality depends on the independence approximation for component forecasts, so calibration should be monitored in deployments where component errors are correlated."],"supporting_citations":[{"why":"Supplies Lag-Llama, the frozen transformer forecaster whose 100-draw predictive samples drive bagging, stacking, and residual correction.","marker":"[Rasul et al., 2023]"},{"why":"Supplies AutoGluon, the automated forecasting and ensemble toolkit used for stacking, residual modeling, and the statistical baseline.","marker":"[Shchur et al., 2023]"},{"why":"Supplies TimeGPT, the encoder-decoder foundation model used for the iterative residual-feedback experiments.","marker":"[Garza and Mergenthaler-Canseco, 2023]"}],"fun_headline_variants":["Bagging and stacking tame time-series foundation models","Residual modeling cuts forecast error 67%","Statistical ensembles boost transformer forecasters","Hybrid models beat standalone time-series AI","Ensemble tricks make foundation models forecast better"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Residual correction (Algorithm 3) trains AutoGluon on the error series and then evaluates on the same rolling window, so the reported 67% three-week gain depends on the unstated assumption that the error model never sees the evaluation period; if that assumption fails, the gain is leakage rather than genuine bias correction.","fun_headline_variants_meta":{"raw":{"variants":["Bagging and stacking tame time-series foundation models","Residual modeling cuts forecast error 67%","Statistical ensembles boost transformer forecasters","Hybrid models beat standalone time-series AI","Ensemble tricks make foundation models forecast better"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000155,"raw_usage":{"total_tokens":1212,"prompt_tokens":943,"completion_tokens":269,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":203}},"tokens_in":559,"tokens_out":269,"duration_ms":3248,"temperature":1.0,"reasoning_tokens":203,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:21:59.849347+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Algorithm 3 with a strict chronological split: fit the AutoGluon error model only on errors strictly before the evaluation window and then score the adjusted forecasts on the held-out period. If the three-week MSE improvement drops far below the reported 67% or turns negative, the residual-correction result is an artifact of look-ahead.","supporting_citations":[{"cited_title":"Lag-llama: Towards foundation models for time series forecasting","cited_arxiv_id":null,"evidence_quote":"Supplies Lag-Llama, the frozen transformer forecaster whose 100-draw predictive samples drive bagging, stacking, and residual correction."},{"cited_title":"Autogluon--timeseries: Automl for probabilistic time series forecasting","cited_arxiv_id":null,"evidence_quote":"Supplies AutoGluon, the automated forecasting and ensemble toolkit used for stacking, residual modeling, and the statistical baseline."}],"review_version":1}