Echo State Networks for Time Series Forecasting: Hyperparameter Sweep and Benchmarking

This paper investigates the performance of Echo State Networks (ESNs) for univariate forecasting of monthly and quarterly time series from the M4 Forecasting Competition dataset. We evaluate whether a simple first-order autoregressive ESN can serve as a competitive alternative to widely used forecasting methods. The study uses a two-stage design: a Parameter dataset is used to analyze ESN model configurations over leakage rate, spectral radius, reservoir size, and regularization selection, while a disjoint Forecast dataset is reserved for out-of-sample benchmarking. Forecast accuracy is measured using mean absolute scaled error (MASE) and symmetric mean absolute percentage error (sMAPE) and compared with simple benchmarks and statistical models including autoregressive integrated moving average (ARIMA), exponential smoothing state space (ETS), the Theta method, and TBATS. The model-configuration analysis reveals frequency-specific patterns: monthly series tend to favor moderately persistent reservoirs, whereas quarterly series favor more contractive dynamics; across both frequencies, high leakage rates are generally preferred. In the final benchmark, the ESN performs on par with ARIMA and TBATS for monthly data and achieves the lowest mean MASE for quarterly data, although it is not uniformly best across all metrics. Overall, the results indicate that a simple autoregressive ESN can provide competitive forecast accuracy on the considered filtered M4 subsets, particularly under MASE, while requiring low training and forecasting time once the ESN configuration has been fixed.