Pith. sign in

REVIEW 3 major objections 4 minor 54 references

The cost of ensembling: is it always worth combining?

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Ensembling forecasting models pays off fast and then stops: two or three models capture most of the gain, and retraining them less often costs little in point accuracy.

desk verdict The retraining-cost findings are solid and practical; the headline accuracy claim rests on oracle-selected ensembles and is overstated. read the letter →

arxiv 2506.04677 v2 pith:UPC3CS53 submitted 2025-06-05 cs.LG stat.APstat.OT

Marco Zanotti This is my paper · ORCID
classification cs.LGstat.APstat.OT
keywords timeseriesforecastingensemblelearningglobalmodelsretrainingfrequencyforecastcombinationM5datasetVN1cost-accuracytrade-off
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Ensembles of forecasting models improve accuracy, but the paper argues the benefit saturates quickly and the cost can become disproportionate. On two large retail datasets, M5 and VN1, ten global models and eight ensemble configurations are compared under retraining schemes from continuous to never. The paper finds that accuracy-selected ensembles of two or three models already reach near-optimal point and probabilistic accuracy, that efficiency-selected ensembles are nearly as accurate at far lower cost, and that periodic or even no retraining preserves point accuracy almost completely. If this is right, practitioners can cut cloud computing and energy costs sharply without sacrificing forecast quality.

What carries the argument

The central mechanism is the simple average of base-model forecasts, and of their quantiles, over pools of global models selected by one of two criteria: accuracy-selected ensembles take the top two to five models by test-window RMSSE rank, while efficiency-selected ensembles take the cheapest models by training and inference time. Retraining scenarios $r$ state how many new observations must accumulate before the pool is re-estimated, ranging from $r=1$ (continuous retraining) to $r=T$ (no retraining). Accuracy is measured by RMSSE and scaled multi-quantile loss, computing time is measured directly, and costs are obtained by converting that time into cloud dollars under stated pricing assumptions. Conformal inference produces the prediction intervals that feed the quantile-averaged probabilistic forecasts.

What would settle it

Rebuild the accuracy-selected ensembles using only validation-window RMSSE ranks computed before the test window; if two- or three-model ensembles no longer beat the best single base model on RMSSE and SMQL, the central accuracy conclusion fails. Alternatively, inject synthetic concept drift into a stable retail series and check whether infrequent retraining loses the near-parity it shows here.

Watch

Extended reading notes

Core claim

On the paper's own terms, ensembling global forecasting models is usually worth it for small pools and for probabilistic forecasts, but not unconditionally. Accuracy-selected ensembles, whose members are the best base models on the test window, improve RMSSE and scaled quantile loss over individual models; adding a third, fourth, or fifth model yields little further gain and can even degrade the ensemble. Efficiency-selected ensembles, built from the fastest models, approach that accuracy on the M5 dataset at roughly half the cost or less. Reducing retraining frequency from continuous to periodic intervals lowers computing time nearly exponentially, and on these stable retail datasets point accuracy is virtually unaffected, while probabilistic accuracy degrades modestly. The paper concludes that monthly retraining is a practical compromise and that small, deliberately selected ensembles with periodic retraining deliver accurate, sustainable forecasts.

Load-bearing premise

The accuracy-driven ensembles are selected on the test window itself, and the retraining results assume the data-generating process stays stable with no concept drift.

Editorial extensions

If this is right

  • Ensembles of two or three accuracy-selected models reach near-optimal RMSSE and scaled quantile loss, and larger pools add cost without proportional accuracy gains.
  • Moving from continuous to the first periodic retraining interval cuts computing time by roughly a third on M5 and nearly half on VN1, with point accuracy essentially unchanged.
  • Efficiency-selected ensembles can undercut accuracy-selected ensembles' cost by more than half while staying competitive, especially on the larger M5 dataset.
  • Probabilistic forecasts benefit more from ensembling than point forecasts do, but they are also the component that degrades first when retraining becomes infrequent.
  • On stable data, even the no-retraining scenario performs nearly as well as continuous retraining for point forecasts, extending earlier results for single global models to ensembles.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is whether a deployable validation-based selection rule reproduces the accuracy-selected ensemble gains, since the paper evaluates those ensembles a posteriori using test-window ranks.
  • The retraining results assume a stable data-generating process without concept drift, so domains with strong seasonality shifts or drift may show larger accuracy losses from infrequent retraining than these retail datasets do.
  • The dollar figures assume fixed cloud pricing, so the quantitative savings are indicative rather than guaranteed under different vendors, parallelism, or hardware choices.
  • Because simple averaging is the cheapest combination rule, the reported costs are effectively a lower bound for ensemble strategies that use weighted combinations or stacking.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper reports an empirical study of ensemble forecasting in global time series models. Ten base learners and eight ensemble configurations (ENSACC, selected by accuracy, and ENSTIME, selected by runtime) are evaluated on the M5 daily and VN1 weekly retail datasets under ten retraining frequencies. The metrics are RMSSE, SMQL, and computing time, with a cloud-cost extrapolation. The main findings are that small ensembles of two or three models approach the best achievable accuracy, that ENSACC ensembles are expensive, that ENSTIME offers a cost-accuracy middle ground, and that reducing retraining frequency from continuous or weekly to monthly or no retraining cuts computing time by large fractions with little point-accuracy loss.

Significance. If the accuracy claims survive scrutiny, the cost-accuracy frontier and retraining-frequency evidence would give practitioners concrete guidance. The retraining portion is well supported by the supplementary tables: M5 RMSSE moves by at most about 0.001 for ENSACC across retraining scenarios, and CT drops by roughly 33% to more than 75% as retraining becomes less frequent. The ENSACC accuracy component, however, is built on test-window oracle selection, so the paper's central accuracy claims are overstated. A revision that reframes ENSACC as a retrospective upper bound or replaces it with a feasible validation-based selection rule would make the contribution sound and useful.

major comments (3)
  1. [3.2.1 and Table 4] ENSACC ensemble membership is selected using RMSSE computed over the full test window, and the text explicitly describes the simulation as 'a posteriori'. This makes the reported ENSACC accuracy an oracle upper bound rather than an achievable accuracy-driven strategy, because a practitioner would not know which base models to include before seeing the test period. The abstract and Section 5 claims that accuracy-optimized ensembles 'consistently improve' forecasting performance therefore rest on circular model selection. Please either introduce a validation-window-based member selection rule or explicitly reframe ENSACC as a retrospective bound, and revise the abstract and conclusions accordingly.
  2. [4, Table 2 and Table 5] Even under the oracle selection, the claim that 'ENSACC ensembles consistently outperform individual base models' is contradicted by the paper's own results. On M5, XGBoost alone has RMSSE 0.739, while Ens2A and Ens3A both have 0.741; on VN1, MLP alone has SMQL 0.806, below Ens2A's 0.826. Thus the accuracy advantage of ENSACC is dataset- and metric-specific rather than consistent, and it is often within 0.002 to 0.02 of the best base model. The text should be rewritten to report which model/metric combinations actually favor ENSACC, and the abstract's 'consistently improve' should be qualified.
  3. [5, Limitations; Figures 1-3] The retraining recommendation—reducing retraining frequency without compromising point forecast accuracy—is explicitly conditional on the absence of concept drift, a condition the paper acknowledges in Section 5 as a limitation but does not carry into the abstract or the practical guidelines. With only two retail datasets and a one-year test window, the near-lossless retraining result is at best a no-drift benchmark. Please state this condition alongside the headline finding and, if possible, add a stress test on a dataset with trend or level shifts to bound the effect; otherwise the practitioner guideline is overgeneralized.
minor comments (4)
  1. [3.4] The quantile list contains '0.025' twice and omits '0.250'; the intended set of 14 levels should be corrected.
  2. [3.4 and Supplementary material] The main text announces conformal inference only in the Supplementary material, yet SMQL results depend on the conformal calibration sets; a brief explanation of how quantiles are produced and calibrated belongs in Section 3.4.
  3. [Abstract] The abstract contains a typo: 'To asses it' should be 'To assess it'.
  4. [Table 2] Table 2 states that minimum RMSSE and SMQL values are highlighted in bold, but the table as rendered has no bold entries; please ensure the formatting is visible in the published version.

Circularity Check

1 steps flagged · score 6.0 of 10

ENSACC accuracy is selected on the same test window used for evaluation, so the headline 'ensembles consistently improve' is an oracle-bound claim.

  1. self definitional [Section 3.2.1 (Ensemble learning) and Section 4 (Table 2 discussion)]
    "In our study, we simulate various ensemble strategies to assess, a posteriori, how forecast performance over the full test window would have changed if forecasting combinations had been used in place of the individual base models. Hence, we employed simple mean ensembles to combine the forecasts of multiple pools of global models, following two distinct selection criteria. The first strategy, which we refer to as ENSACC, combines the base models that demonstrated the highest individual point forecast accuracy (measured by Equation 4)."

    ENSACC is defined as the subset of base models with the best RMSSE (Equation 4) computed over the full test window. The paper then reports RMSSE for Ens2A–Ens5A on that same test window (Table 2) and concludes, in Section 4, that 'ENSACC ensembles consistently outperform individual base models.' Because the ensemble members are selected by their test-window RMSSE rank and evaluated with the same RMSSE on the same window, the reported point-accuracy advantage is built into the construction: averaging the top-RMSSE models and then measuring RMSSE is an oracle-selected upper bound, not an independent prediction.

full rationale

The central circular step is the ENSACC construction: its member models are chosen a posteriori by RMSSE over the full test window (Section 3.2.1), and then the same RMSSE metric on the same test window is used to claim that these ensembles consistently beat individual models (Section 4, Table 2). This makes the accuracy gain a consequence of the selection rule rather than an independently verified prediction. Notably, even under this oracle selection, Table 2 shows XGBoost alone (RMSSE 0.739) beats Ens2A and Ens3A (RMSSE 0.741) on M5, so the paper's own numbers contradict the unqualified claim that ENSACC consistently outperforms individual base models. The retraining-frequency and cost conclusions are substantially more independent: they compare relative RMSSE, SMQL, and computing time across retraining scenarios for fixed ensemble compositions, so those findings do not reduce to the selection criterion. The self-citation to Zanotti (2025) for retraining scenarios and computing-time definition is not load-bearing because the current experiments compute these quantities directly. Overall, one central 'prediction' reduces by construction, giving a partial circularity score of 6 rather than a full collapse.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central empirical claims rest on six hand-chosen settings (cost rate, retailer scale, ensemble cap, baseline scenario, validation multipliers, base hyperparameters) and five domain assumptions. No new theoretical entities are introduced. The most consequential choice is the a posteriori definition of ENSACC ensembles, which is captured in the red flags rather than the ledger.

free parameters (6)
  • Cloud cost rate = $3.5/hour
    Assumed price per hour of Azure NC6s v3 (Section 4, Figure 4). All dollar cost estimates scale linearly with this rate.
  • Retailer scale = 200,000 SKUs, 5,000 stores
    Used to extrapolate per-series cost to a large retailer; creates the multi-million dollar cost figures in Figures 4-7.
  • Ensemble size cap = 5 models
    ENSACC and ENSTIME ensembles include the top 2 to 5 models; the cap bounds the study and shapes the 'small ensembles suffice' conclusion.
  • Baseline retraining scenarios = r=7 (M5), r=1 (VN1)
    All relative accuracy and cost metrics are normalized to this baseline, so the choice of benchmark drives the reported percentages.
  • Validation set multiplier for conformal calibration = 4x horizon (M5), 2x horizon (VN1)
    Used in the supplementary conformal inference setup; determines which series are retained and the probabilistic forecasts.
  • Base model hyperparameters = Competition entries or library defaults
    Section 3.2 states hyperparameters come from leading competition solutions or default values; exact settings are not listed, and they drive both accuracy and cost.
assumptions (5)
  • domain assumption The data-generating process in M5 and VN1 is stable, without notable trends or concept drift
    Acknowledged as a limitation in Section 5: 'this study operates under the assumption that the data-generating process remains stable.' It underpins the conclusion that infrequent retraining preserves accuracy.
  • domain assumption Conformal inference produces valid prediction intervals for the retained time series
    Used in the supplementary to generate quantile forecasts; assumes exchangeability or its time series adaptation from Stankeviciute et al. (2021) and Vovk et al. (2005).
  • domain assumption RMSSE and SMQL are appropriate accuracy measures for intermittent retail demand
    Section 3.4 motivates these metrics using Kolassa (2020); the metric choice determines all accuracy rankings.
  • domain assumption Computing time on one Azure NC6s v3 instance is a valid proxy for production forecasting cost
    Section 3.4 defines CT and Section 4 multiplies it by a fixed hourly price; no parallelization, vendor variation, or energy cost is modeled.
  • standard math Friedman-Nemenyi test provides valid multiple comparisons for model ranking
    Section 3.4 states the tests are used; the procedure is standard but relies on the usual assumptions about paired observations and independent series.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The cost of ensembling: is it always worth combining?." pith.science (2026). https://pith.science/paper/UPC3CS53

@misc{pith2026250604677,
  author       = {Pith},
  title        = {Pith review of: The cost of ensembling: is it always worth combining?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UPC3CS53}},
  note         = {Machine review of arXiv:2506.04677}
}
read the original abstract

Given the continuous increase in dataset sizes and the complexity of forecasting models, the trade-off between forecast accuracy and computational cost is emerging as an extremely relevant topic, especially in the context of ensemble learning for time series forecasting. To asses it, we evaluated ten base models and eight ensemble configurations across two large-scale retail datasets (M5 and VN1), considering both point and probabilistic accuracy under varying retraining frequencies. We showed that ensembles consistently improve forecasting performance, particularly in probabilistic settings. However, these gains come at a substantial computational cost, especially for larger, accuracy-driven ensembles. We found that reducing retraining frequency significantly lowers costs, with minimal impact on accuracy, particularly for point forecasts. Moreover, efficiency-driven ensembles offer a strong balance, achieving competitive accuracy with considerably lower costs compared to accuracy-optimized combinations. Most importantly, small ensembles of two or three models are often sufficient to achieve near-optimal results. These findings provide practical guidelines for deploying scalable and cost-efficient forecasting systems, supporting the broader goals of sustainable AI in forecasting. Overall, this work shows that careful ensemble design and retraining strategy selection can yield accurate, robust, and cost-effective forecasts suitable for real-world applications.

Figures

Figures reproduced from arXiv: 2506.04677 by the authors.

Figure 1
Figure 1. RMSSE results for each ensemble and retrain scenario combination in relative terms with respect to the [PITH_FULL_IMAGE:figures/full_fig_p018_1.png] view at source ↗
Figure 2
Figure 2. SMQL results for each ensemble and retrain scenario combination in relative terms with respect to the [PITH_FULL_IMAGE:figures/full_fig_p019_2.png] view at source ↗
Figure 3
Figure 3. CT results for each ensemble and retrain scenario combination in relative terms with respect to the baseline [PITH_FULL_IMAGE:figures/full_fig_p020_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Cost analysis. Overall cost results for the M5 and VN1 datasets. [PITH_FULL_IMAGE:figures/full_fig_p022_4.png]
Figure 5
Figure 5. Figure 5: Cost analysis. RMSSE vs Cost ($) for the M5 and VN1 datasets. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_5.png]
Figure 6
Figure 6. Figure 6: Cost analysis. SMQL vs Cost ($) for the M5 and VN1 datasets. (see Supplementary material). In contrast, the M5 dataset exhibits steady, incremental reductions in cost as retraining becomes less frequent. For example, in the M5 setting, the average cost of continuous re…
Figure 7
Figure 7. Figure 7: Cost analysis. Comparison between base models and ensembles for the M5 dataset. Each method and [PITH_FULL_IMAGE:figures/full_fig_p024_7.png]
Figure 8
Figure 8. Figure 8: Friedman-Nemenyi test results based on RMSSE. The baseline scenarios are [PITH_FULL_IMAGE:figures/full_fig_p038_8.png]
Figure 9
Figure 9. Figure 9: Friedman-Nemenyi test results based on SMQL. The baseline scenarios are [PITH_FULL_IMAGE:figures/full_fig_p039_9.png]
Figure 10
Figure 10. Figure 10: M5 Friedman-Nemenyi test results based on RMSSE. [PITH_FULL_IMAGE:figures/full_fig_p040_10.png]
Figure 11
Figure 11. Figure 11: VN1 Friedman-Nemenyi test results based on RMSSE. [PITH_FULL_IMAGE:figures/full_fig_p040_11.png]
Figure 12
Figure 12. Figure 12: M5 Friedman-Nemenyi test results based on SMQL. [PITH_FULL_IMAGE:figures/full_fig_p041_12.png]
Figure 13
Figure 13. Figure 13: VN1 Friedman-Nemenyi test results based on SMQL. [PITH_FULL_IMAGE:figures/full_fig_p041_13.png]
Figure 14
Figure 14. Figure 14: Cost analysis. Comparison between base models and ensembles for the VN1 dataset. Each method and [PITH_FULL_IMAGE:figures/full_fig_p044_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 26 canonical work pages

  1. [1]

    , author Bergmeir, C

    author Bandara, K. , author Bergmeir, C. , & author Smyl, S. ( year 2020 ). title Forecasting across time series databases using recurrent neural networks on groups of similar series: A clustering approach . journal Expert Systems with Applications \/ , volume 140 \/ , pages 112896 . https://www.sciencedirect.com/science/article/pii/S0957417419306128. :ht...

    arXiv 2020

  2. [2]

    , author Hewamalage, H

    author Bandara, K. , author Hewamalage, H. , author Liu, Y.-H. , author Kang, Y. , & author Bergmeir, C. ( year 2021 ). title Improving the accuracy of global forecasting models using time series data augmentation . journal Pattern Recognition \/ , volume 120 \/ , pages 108148 . https://www.sciencedirect.com/science/article/pii/S0031320321003356. :https:/...

    work page arXiv 2021

  3. [3]

    author Barrow, D. K. , & author Kourentzes, N. ( year 2016 ). title Distributions of forecasting errors of forecast combinations: Implications for inventory management . journal International Journal of Production Economics \/ , volume 177 \/ , pages 24--33 . https://www.sciencedirect.com/science/article/pii/S0925527316300226. :https://doi.org/10.1016/j.i...

    work page doi:10.1016/j.ijpe.2016.03.017 2016

  4. [4]

    , & author Benítez, J

    author Bergmeir, C. , & author Benítez, J. M. ( year 2012 ). title On the use of cross-validation for time series predictor evaluation . journal Information Sciences \/ , volume 191 \/ , pages 192--213 . https://www.sciencedirect.com/science/article/pii/S0020025511006773. :https://doi.org/10.1016/j.ins.2011.12.028. note Data Mining for Software Trustworthiness

    doi:10.1016/j.ins.2011.12.028 2012

  5. [5]

    ( year 2001 )

    author Breiman, L. ( year 2001 ). title Random forests . journal Machine Learning \/ , volume 45 \/ , pages 5--32 . :10.1023/A:1010933404324

    doi:10.1023/a:1010933404324 2001

  6. [6]

    ( year 2017 )

    author Busetti, F. ( year 2017 ). title Quantile aggregation of density forecasts . journal Oxford Bulletin of Economics and Statistics \/ , volume 79 \/ , pages 495--512 . https://onlinelibrary.wiley.com/doi/abs/10.1111/obes.12163. :https://doi.org/10.1111/obes.12163. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/obes.12163 http://arxiv.org/abs/h...

    work page doi:10.1111/obes.12163 2017

  7. [7]

    , author Olivares, K

    author Challu, C. , author Olivares, K. G. , author Oreshkin, B. N. , author Garza, F. , author Mergenthaler-Canseco, M. , & author Dubrawski, A. W. ( year 2022 ). title N-hits: Neural hierarchical interpolation for time series forecasting . journal ArXiv \/ , volume abs/2201.12886 \/ . https://api.semanticscholar.org/CorpusID:246430557

    arXiv 2022

  8. [8]

    , & author Guestrin, C

    author Chen, T. , & author Guestrin, C. ( year 2016 ). title Xgboost: A scalable tree boosting system . In booktitle Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining \/ KDD '16 (p. pages 785–794 ). address New York, NY, USA : publisher Association for Computing Machinery . https://doi.org/10.1145/2939672.2...

    arXiv 2016

Show all 54 references
  1. [9]

    , author Magnus, J

    author Claeskens, G. , author Magnus, J. R. , author Vasnev, A. L. , & author Wang, W. ( year 2016 ). title The forecast combination puzzle: A simple theoretical explanation . journal International Journal of Forecasting \/ , volume 32 \/ , pages 754--762 . https://www.science...

    2016 doi

  2. [10]

    ( year 2006 )

    author Dem s ar, J. ( year 2006 ). title Statistical comparisons of classifiers over multiple data sets . journal Journal of Machine Learning Research \/ , volume 7 \/ , pages 1--30 . http://jmlr.org/papers/v7/demsar06a.html

    2006

  3. [11]

    , author Ma, S

    author Fildes, R. , author Ma, S. , & author Kolassa, S. ( year 2022 ). title Retail forecasting: Research and practice . journal International Journal of Forecasting \/ , volume 38 \/ , pages 1283--1318 . https://www.sciencedirect.com/science/article/pii/S016920701930192X. :h...

    2022 doi

  4. [12]

    S., Yael Grushka-Cockayne , & author Spiliotis, E

    author Fotios Petropoulos, E. S., Yael Grushka-Cockayne , & author Spiliotis, E. ( year 2024 ). title Wielding occam’s razor: Fast and frugal retail forecasting . journal Journal of the Operational Research Society \/ , volume 0 \/ , pages 1--20 . https://doi.org/10.1080/01605...

    2024

  5. [13]

    , author Charpentier, B

    author Getzner, J. , author Charpentier, B. , & author Günnemann, S. ( year 2023 ). title Accuracy is not the only metric that matters: Estimating the energy consumption of deep learning models . https://arxiv.org/abs/2304.00897. arXiv:2304.00897 http://arxiv.org/abs/2304.00897

    2023 arXiv

  6. [14]

    , author Bandara, K

    author Godahewa, R. , author Bandara, K. , author Webb, G. I. , author Smyl, S. , & author Bergmeir, C. ( year 2021 a ). title Ensembles of localised models for time series forecasting . journal Knowledge-Based Systems \/ , volume 233 \/ , pages 107518 . https://www.sciencedir...

    2021

  7. [15]

    author Godahewa, R. W. , author Bergmeir, C. , author Webb, G. I. , author Hyndman, R. , & author Montero-Manso, P. ( year 2021 b ). title Monash time series forecasting archive . In booktitle Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchm...

    2021

  8. [16]

    , author Ackermann, K

    author Hewamalage, H. , author Ackermann, K. , & author Bergmeir, C. ( year 2023 ). title Forecast evaluation for data scientists: common pitfalls and best practices . journal Data Mining and Knowledge Discovery \/ , volume 37 \/ , pages 788--832 . :10.1007/s10618-022-00894-5

    2023 doi

  9. [17]

    , & author Schmidhuber, J

    author Hochreiter, S. , & author Schmidhuber, J. ( year 1997 ). title Long short-term memory . journal Neural Comput. \/ , volume 9 \/ , pages 1735–1780 . https://doi.org/10.1162/neco.1997.9.8.1735. :10.1162/neco.1997.9.8.1735

    1997 doi

  10. [18]

    , & author Makridakis, S

    author Howard, A. , & author Makridakis, S. ( year 2020 ). title M5 forecasting - accuracy . howpublished https://kaggle.com/competitions/m5-forecasting-accuracy . note Kaggle

    2020

  11. [19]

    , & author Stuckenschmidt, H

    author Huber, J. , & author Stuckenschmidt, H. ( year 2020 ). title Daily retail demand forecasting using machine learning with emphasis on calendric special days . journal International Journal of Forecasting \/ , volume 36 \/ , pages 1420--1438 . https://www.sciencedirect.co...

    2020 doi

  12. [20]

    author Hyndman, R. J. , & author Koehler, A. B. ( year 2006 ). title Another look at measures of forecast accuracy . journal International Journal of Forecasting \/ , volume 22 \/ , pages 679--688 . https://www.sciencedirect.com/science/article/pii/S0169207006000239. :https://...

    2006 doi

  13. [21]

    , author Gasthaus, J

    author Januschowski, T. , author Gasthaus, J. , author Wang, Y. , author Salinas, D. , author Flunkert, V. , author Bohlke-Schneider, M. , & author Callot, L. ( year 2020 ). title Criteria for classifying forecasting methods . journal International Journal of Forecasting \/ , ...

    2020 doi

  14. [22]

    , author Wang, Y

    author Januschowski, T. , author Wang, Y. , author Torkkola, K. , author Erkkilä, T. , author Hasson, H. , & author Gasthaus, J. ( year 2022 ). title Forecasting with trees . journal International Journal of Forecasting \/ , volume 38 \/ , pages 1473--1481 . https://www.scienc...

    2022 doi

  15. [23]

    , author Meng, Q

    author Ke, G. , author Meng, Q. , author Finley, T. , author Wang, T. , author Chen, W. , author Ma, W. , author Ye, Q. , & author Liu, T.-Y. ( year 2017 ). title Lightgbm: A highly efficient gradient boosting decision tree . In editor I. Guyon , editor U. V. Luxburg , editor ...

    2017

  16. [24]

    ( year 2011 )

    author Kolassa, S. ( year 2011 ). title Combining exponential smoothing forecasts using akaike weights . journal International Journal of Forecasting \/ , volume 27 \/ , pages 238--251 . https://www.sciencedirect.com/science/article/pii/S0169207010001032. :https://doi.org/10.1...

    2011 doi

  17. [25]

    ( year 2016 )

    author Kolassa, S. ( year 2016 ). title Evaluating predictive count data distributions in retail sales forecasting . journal International Journal of Forecasting \/ , volume 32 \/ , pages 788--803 . https://www.sciencedirect.com/science/article/pii/S0169207016000315. :https://...

    2016 doi

  18. [26]

    ( year 2020 )

    author Kolassa, S. ( year 2020 ). title Why the “best” point forecast depends on the error or accuracy measure . journal International Journal of Forecasting \/ , volume 36 \/ , pages 208--211 . https://www.sciencedirect.com/science/article/pii/S0169207019301359. :https://doi....

    2020 doi

  19. [27]

    , author Barrow, D

    author Kourentzes, N. , author Barrow, D. , & author Petropoulos, F. ( year 2019 ). title Another look at forecast selection and combination: Evidence from forecast pooling . journal International Journal of Production Economics \/ , volume 209 \/ , pages 226--235 . https://ww...

    2019 doi

  20. [28]

    author Lainder, A. D. , & author Wolfinger, R. D. ( year 2022 ). title Forecasting with gradient boosted trees: augmentation, tuning, and cross-validation strategies: Winning solution to the m5 uncertainty competition . journal International Journal of Forecasting \/ , volume ...

    2022 doi

  21. [29]

    , author Spiliotis, E

    author Makridakis, S. , author Spiliotis, E. , & author Assimakopoulos, V. ( year 2022 a ). title M5 accuracy competition: Results, findings, and conclusions . journal International Journal of Forecasting \/ , volume 38 \/ , pages 1346--1364 . https://www.sciencedirect.com/sci...

    2022 doi

  22. [30]

    , author Spiliotis, E

    author Makridakis, S. , author Spiliotis, E. , & author Assimakopoulos, V. ( year 2022 b ). title The m5 competition: Background, organization, and implementation . journal International Journal of Forecasting \/ , volume 38 \/ , pages 1325--1336 . https://www.sciencedirect.co...

    2022 doi

  23. [31]

    , author Spiliotis, E

    author Makridakis, S. , author Spiliotis, E. , author Assimakopoulos, V. , author Chen, Z. , author Gaba, A. , author Tsetlin, I. , & author Winkler, R. L. ( year 2022 c ). title The m5 uncertainty competition: Results, findings and conclusions . journal International Journal ...

    2022 doi

  24. [32]

    ( year 2023 )

    author Montero-Manso, P. ( year 2023 ). title How to leverage data for time series forecasting with artificial intelligence models: Illustrations and guidelines for cross-learning . In editor M. Hamoudia , editor S. Makridakis , & editor E. Spiliotis (Eds.), booktitle Forecast...

    2023 doi

  25. [33]

    , & author Hyndman, R

    author Montero-Manso, P. , & author Hyndman, R. J. ( year 2021 ). title Principles and algorithms for forecasting groups of time series: Locality and globality . journal International Journal of Forecasting \/ , volume 37 \/ , pages 1632--1653 . https://www.sciencedirect.com/s...

    2021 doi

  26. [34]

    , & author Granger, C

    author Newbold, P. , & author Granger, C. W. J. ( year 1974 ). title Experience with forecasting univariate time series and the combination of forecasts . journal Journal of the Royal Statistical Society. Series A (General) \/ , volume 137 \/ , pages 131--165 . http://www.jsto...

    1974

  27. [35]

    , & author Petropoulos, F

    author Nikolopoulos, K. , & author Petropoulos, F. ( year 2018 ). title Forecasting for big data: Does suboptimality matter? journal Computers & Operations Research \/ , volume 98 \/ , pages 322--329 . https://www.sciencedirect.com/science/article/pii/S0305054817301193. :https...

    2018 doi

  28. [36]

    title Nixtla - state-of-the-art time series and forecasting software

    author Nixtla ( year 2022 ). title Nixtla - state-of-the-art time series and forecasting software . howpublished https://nixtlaverse.nixtla.io/ . note Nixtla

    2022

  29. [37]

    , author Dieleman, S

    author Van den Oord, A. , author Dieleman, S. , author Zen, H. , author Simonyan, K. , author Vinyals, O. , author Graves, A. , author Kalchbrenner, N. , author Senior, A. W. , & author Kavukcuoglu, K. ( year 2016 ). title Wavenet: A generative model for raw audio. journal CoR...

    2016 arXiv

  30. [38]

    author Oreshkin, B. N. , author Carpov, D. , author Chapados, N. , & author Bengio, Y. ( year 2020 ). title N-beats: Neural basis expansion analysis for interpretable time series forecasting . In booktitle International Conference on Learning Representations \/ . https://openr...

    2020

  31. [39]

    , & author Chorowska, A

    author Pawlikowski, M. , & author Chorowska, A. ( year 2020 ). title Weighted ensemble of statistical models . journal International Journal of Forecasting \/ , volume 36 \/ , pages 93--97 . https://www.sciencedirect.com/science/article/pii/S0169207019301190. :https://doi.org/...

    2020 doi

  32. [40]

    , & author et al

    author Petropoulos, F. , & author et al. ( year 2022 ). title Forecasting: theory and practice . journal International Journal of Forecasting \/ , volume 38 \/ , pages 705--871 . https://www.sciencedirect.com/science/article/pii/S0169207021001758. :https://doi.org/10.1016/j.ij...

    2022 doi

  33. [41]

    , author Gusev, G

    author Prokhorenkova, L. , author Gusev, G. , author Vorobev, A. , author Dorogush, A. V. , & author Gulin, A. ( year 2018 ). title Catboost: unbiased boosting with categorical features . In booktitle Proceedings of the 32nd International Conference on Neural Information Proce...

    2018

  34. [42]

    ( year 1958 )

    author Rosenblatt, F. ( year 1958 ). title The perceptron: A probabilistic model for information storage and organization in the brain . journal Psychological Review \/ , volume 65 \/ , pages 386--408 . :10.1037/h0042519

    1958 doi

  35. [43]

    , author Dodge, J

    author Schwartz, R. , author Dodge, J. , author Smith, N. A. , & author Etzioni, O. ( year 2020 ). title Green ai . journal Commun. ACM \/ , volume 63 \/ , pages 54–63 . https://doi.org/10.1145/3381831. :10.1145/3381831

    2020 doi

  36. [44]

    , author Spiliotis, E

    author Semenoglou, A.-A. , author Spiliotis, E. , author Makridakis, S. , & author Assimakopoulos, V. ( year 2021 ). title Investigating the accuracy of cross-learning time series forecasting methods . journal International Journal of Forecasting \/ , volume 37 \/ , pages 1072...

    2021 doi

  37. [45]

    ( year 2020 )

    author Smyl, S. ( year 2020 ). title A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting . journal International Journal of Forecasting \/ , volume 36 \/ , pages 75--85 . https://www.sciencedirect.com/science/article/pii/S01692070...

    2020 doi

  38. [46]

    , & author Hua, N

    author Smyl, S. , & author Hua, N. G. ( year 2019 ). title Machine learning methods for gefcom2017 probabilistic load forecasting . journal International Journal of Forecasting \/ , volume 35 \/ , pages 1424--1431 . https://www.sciencedirect.com/science/article/pii/S0169207019...

    2019 doi

  39. [47]

    , & author Petropoulos, F

    author Spiliotis, E. , & author Petropoulos, F. ( year 2024 ). title On the update frequency of univariate forecasting models . journal European Journal of Operational Research \/ , volume 314 \/ , pages 111--121 . https://www.sciencedirect.com/science/article/pii/S03772217230...

    2024 doi

  40. [48]

    , author M

    author Stankeviciute, K. , author M. Alaa, A. , & author van der Schaar, M. ( year 2021 ). title Conformal time-series forecasting . In editor M. Ranzato , editor A. Beygelzimer , editor Y. Dauphin , editor P. Liang , & editor J. W. Vaughan (Eds.), booktitle Advances in Neural...

    2021

  41. [49]

    author Tashman, L. J. ( year 2000 ). title Out-of-sample tests of forecasting accuracy: an analysis and review . journal International Journal of Forecasting \/ , volume 16 \/ , pages 437--450 . https://www.sciencedirect.com/science/article/pii/S0169207000000650. :https://doi....

    2000 doi

  42. [50]

    ( year 2024 )

    author Vandeput, N. ( year 2024 ). title Vn1 forecasting - accuracy challenge . howpublished https://www.datasource.ai/en/home/data-science-competitions-for-startups/phase-2-vn1-forecasting-accuracy-challenge/description . note DataSource.ai

    2024

  43. [51]

    , author Gammerman, A

    author Vovk, V. , author Gammerman, A. , & author Shafer, G. ( year 2005 ). title Algorithmic Learning in a Random World \/ . address Berlin, Heidelberg : publisher Springer-Verlag

    2005

  44. [52]

    , author Hyndman, R

    author Wang, X. , author Hyndman, R. J. , author Li, F. , & author Kang, Y. ( year 2023 ). title Forecast combinations: An over 50-year review . journal International Journal of Forecasting \/ , volume 39 \/ , pages 1518--1547 . https://www.sciencedirect.com/science/article/pi...

    2023 doi

  45. [53]

    ( year 2025 )

    author Zanotti, M. ( year 2025 ). title Do global forecasting models require frequent retraining? https://arxiv.org/abs/2505.00356. arXiv:2505.00356 http://arxiv.org/abs/2505.00356

    2025

  46. [54]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.