REVIEW 4 major objections 5 minor 18 references
A Review of the Long Horizon Forecasting Problem in Time Series Analysis
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Forecast errors grow with horizon length—except in two architectures
desk verdict A useful survey of long-horizon forecasting that overreaches on its empirical error-propagation claim, which is confounded by a nonstandard xLSTM variant and test-set-driven hyperparameter selection. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the per-timestep MSE heatmap: for each test series, the model's squared error at every position of the forecast horizon is averaged across the test set, producing a curve of error versus horizon distance. For most models that curve rises with horizon length; for xLSTM and Triformer it stays comparatively flat. The argument works by contrasting these curves across model families and attributing the flat exceptions to architectural mechanisms: Triformer's factorized variable-specific attention over triangular patches, and xLSTM's strided-convolution input processing with scalar and matrix recurrent memory. The heatmap is what turns the otherwise standard benchmark scores into a claim about error propagation.
What would settle it
Re-run the same ablation study on ETTm2 with all hyperparameters chosen using only the validation split, then recompute the per-timestep MSE heatmaps on the untouched test set; if xLSTM and Triformer still show non-growing error while the other models grow, the error-propagation claim is supported, and if not, the exceptional behavior was an artifact of configuration search.
Extended reading notes
Core claim
The paper claims that long-horizon forecasting should be viewed as an error propagation problem. On ETTm2, the authors compute, for each time step along the forecast horizon, the MSE averaged over the test set, and find that error rises steadily with distance for the MLP, self-attention, and pyramidal self-attention models under study. The exceptions are xLSTM and Triformer, whose per-step errors do not grow with horizon length. Triformer's hierarchical triangular attention distributes the forecast across scales, while xLSTM's scalar and matrix memory states with exponential gating appear to reverse the dependence on length. The paper motivates this pattern as evidence that LHF is governed by error accumulation, and offers the heatmap representation as the diagnostic that exposes it.
Load-bearing premise
The reported comparisons and the exceptional flat-error behavior of xLSTM and Triformer assume that the hyperparameter search, which selected configurations for good MAE without a described validation protocol, did not overfit the test set.
Editorial extensions
If this is right
- If error propagation is the binding constraint, then horizon-length error curves, not aggregate MSE alone, should be the primary diagnostic for LHF models.
- Architectures that escape the rising-error pattern—hierarchical attention and recurrent memory with stabilized gating—become the natural templates for designing longer-horizon models.
- The survey supports the earlier finding that transformer-style attention does not automatically solve LHF on this benchmark, since the rising-error pattern appears across the self-attention family too.
- A model that cannot keep per-step error flat will continue to degrade as the horizon grows, even if its aggregate score is competitive at short horizons.
Reading between the lines
- The error-propagation claim rests on one dataset (ETTm2); a straightforward extension would be to recompute these per-step heatmaps on electricity, traffic, and weather benchmarks to see whether the rising-error curve is a general law or an ETTm2 artifact.
- Because the hyperparameters were tuned on MAE without a stated validation protocol, the flat heatmaps of xLSTM and Triformer need a robustness check: repeat the search with validation-based early stopping and observe whether the flat pattern survives.
- The paper's own note that NHITS also shows a similar flat pattern in the multivariate heatmap hints that the real mechanism is multi-resolution pooling rather than recurrence per se, which would broaden the design space beyond the two named exceptions.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper is a review of long-horizon forecasting (LHF) in time series analysis, covering statistical and deep-learning approaches over roughly the last 35 years. It surveys decomposition techniques, windowing schemes, preprocessing methods, and model families (MLPs, RNNs, self-attention, and pyramidal self-attention), and it reports ablation experiments on the ETTm2 dataset in both multivariate and univariate (HUFL) settings for horizons H = 96, 192, 336, 720. The authors use a nonstandard 16/4/4 month train/validation/test split, compare their results to those in the cited literature, and present per-timestep MSE heatmaps. The central empirical claim, stated in the abstract, is that per-timestep MSE increases steadily with horizon length for most models except xLSTM and Triformer, which the authors interpret as motivating LHF as an error-propagation problem.
Significance. If the central claim were properly supported, the paper would make a useful contribution: it consolidates a large literature, provides a consistent ablation across several model families on a common benchmark, and proposes a concrete mechanistic interpretation of long-horizon degradation. The public release of trained models and the systematic comparison with cited results (including the 7:1:2 split delta in Table 1) are commendable and increase the reproducibility of the survey portion. However, the load-bearing empirical claim is currently supported only by visual inspection of heatmaps from single runs, with no numerical per-timestep curves, no uncertainty quantification, a hyperparameter-selection protocol that may involve test-set feedback, and an xLSTM variant whose strided-convolution implementation is acknowledged by the authors to be visible in the heatmap. These issues make the paper's main interpretation premature.
major comments (4)
- [§8.1, Table 4] The central claim of the paper, that per-timestep MSE increases steadily with horizon for all models except xLSTM and Triformer, rests entirely on visual inspection of the heatmaps in Table 4. The per-timestep curves are not published numerically, no error bars or repeated seeds are reported, and no statistical test is applied to the monotonicity or to the proposed exceptions. The abstract states this pattern as a finding, but the evidence is a single-run visual pattern. Please report the numerical per-step MSE curves averaged over multiple seeds with confidence intervals, and quantify the monotonic increase and the exceptions (e.g., slope estimates or a contrast test) so that the claim is falsifiable.
- [§7.4, §8.1] The hyperparameter configurations are described in Section 7.4 as those 'that worked well with respect to the Mean Average Error (MAE),' with no validation protocol stated, and Section 8.1 says the xLSTM reversal appears 'when the models find the best hyperparameters over the ETTm2 test set.' If test-set performance was used to select configurations, the reported comparisons and the xLSTM/Triformer exception are confounded by test-set feedback. Please state the exact selection protocol and, if the test set was used, rerun the experiments under a strict train/validation/test protocol and confirm that the heatmap patterns persist.
- [§7.4, §8.1] The xLSTM exception cannot currently be attributed to xLSTM's recurrent memory: the implementation uses a strided convolution 'of length equal to the kernel size and repeated features to maintain the length' (Section 7.4), and Section 8.1 states that this implementation 'is visible in its multivariate forecasting heatmap... corresponding to the stride value.' A stride-induced periodic feature-repetition pattern can produce a non-monotonic per-step error curve that is unrelated to error propagation or memory. Please run the canonical xLSTM (or an ablation without the strided convolution) and show that the flat or non-monotonic pattern persists.
- [§7.2, §8.1] The empirical support for the 'error propagation' framing is based on a single dataset (ETTm2) with a nonstandard 16/4/4 split (Section 7.2). Even if the heatmap patterns were confirmed, the claim that LHF is 'an error propagation problem' as a general characterization would require evidence across datasets and horizons. As written, the paper offers an interpretation of one dataset's heatmaps. Please either add additional datasets or explicitly restrict the claim to ETTm2 and describe it as a hypothesis rather than a general finding.
minor comments (5)
- [Table 1] The Δ column uses nonstandard symbols (✓, ×, downward arrows) that are only partially explained in the caption; please make the notation fully self-contained, including what the downward arrow means for the entries where no prior score is cited.
- [Table 4] The heatmaps are not reproducible from the text: the table reports only aggregate MSE values with embedded images, and the underlying per-timestep arrays are not provided. Please make the numerical per-step error curves available, for example in the model repository or in an appendix.
- [§6.2.3] In Equation (10), the stabilization state mt and the exponential gates are introduced, but the superscript convention for the stabilized input gate (i^s_t) is not defined before its first use, which makes the equation hard to parse for a reader not already familiar with the xLSTM paper.
- [§8.3] The sentence on State Space Models contains a subject–verb disagreement ('has number of parameters'), and the referenced SpaceTime results are mentioned without a direct comparison in the ablation tables; please either add the comparison or clearly mark it as external.
- [Abstract and §1] The phrase 'over the last 35 years or so' is informal for a journal abstract, and the paper uses 'this review' to refer to both the present article and the literature it covers in several places; please use 'this paper' for the present work to avoid ambiguity.
Circularity Check
No significant circularity: the heatmap-based LHF framing is an empirical interpretation, not a derivation that reduces to its inputs.
full rationale
This paper is primarily a literature review supplemented by ETTm2 ablation experiments. The central claim, that per-timestep MSE rises with horizon length except for xLSTM and Triformer, is presented as a direct empirical observation from heatmaps of test-set errors (Section 8.1, Table 4), not as a quantity derived from a fitted formula, normalization, or self-citation. The paper does not construct equations in which the claimed output is equivalent to its input by definition, and it does not invoke any load-bearing uniqueness theorem or ansatz from the authors' prior work. The reported confounds, such as the strided-convolution implementation of xLSTM and the statement that hyperparameters were chosen to perform well on the ETTm2 test set, are threats to the validity or generalizability of the empirical comparison, but they do not make the empirical claim circular in the sense of reducing to its own inputs. The review's framing of LHF as an error-propagation problem is an interpretation of observed error behavior, not a derivation forced by construction. No circular steps are present.
Assumptions & free parameters
free parameters (2)
- Per-model hyperparameter configurations =
Not disclosed; selected by MAE
- xLSTM strided convolution kernel settings =
Kernel length equal to the convolution length, with repeated features
assumptions (3)
- domain assumption ETTm2 is a sufficient benchmark for conclusions about the LHF problem.
- domain assumption The custom implementations of the cited models faithfully represent the original architectures.
- ad hoc to paper Visual inspection of per-timestep MSE heatmaps is sufficient to infer an error-propagation mechanism.
Cite this review
Pith. "Pith review of A Review of the Long Horizon Forecasting Problem in Time Series Analysis." pith.science (2026). https://pith.science/paper/LXZ4PWWX
@misc{pith2026250612809,
author = {Pith},
title = {Pith review of: A Review of the Long Horizon Forecasting Problem in Time Series Analysis},
year = {2026},
howpublished = {\url{https://pith.science/paper/LXZ4PWWX}},
note = {Machine review of arXiv:2506.12809}
}
read the original abstract
The long horizon forecasting (LHF) problem has come up in the time series literature for over the last 35 years or so. This review covers aspects of LHF in this period and how deep learning has incorporated variants of trend, seasonality, fourier and wavelet transforms, misspecification bias reduction and bandpass filters while contributing using convolutions, residual connections, sparsity reduction, strided convolutions, attention masks, SSMs, normalization methods, low-rank approximations and gating mechanisms. We highlight time series decomposition techniques, input data preprocessing and dataset windowing schemes that improve performance. Multi-layer perceptron models, recurrent neural network hybrids, self-attention models that improve and/or address the performances of the LHF problem are described, with an emphasis on the feature space construction. Ablation studies are conducted over the ETTm2 dataset in the multivariate and univariate high useful load (HUFL) forecasting contexts, evaluated over the last 4 months of the dataset. The heatmaps of MSE averages per time step over test set series in the horizon show that there is a steady increase in the error proportionate to its length except with xLSTM and Triformer models and motivate LHF as an error propagation problem. The trained models are available here: https://bit.ly/LHFModelZoo
Reference graph
Works this paper leans on
-
[3]
arX iv preprint arXiv:2303.09489
Effectively modeling time series with simple discrete state spaces. arX iv preprint arXiv:2303.09489 . Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., Zh ang, W.,
-
[9]
ar Xiv preprint arXiv:1707.01926
Diffusion convolut ional re- current neural network: Data-driven traffic forecasting. ar Xiv preprint arXiv:1707.01926 . Liang, Y., Wen, H., Nie, Y., Jiang, Y., Jin, M., Song, D., Pan, S., Wen, Q.,
-
[13]
arXiv preprint arXiv:2010.00951
Coupled oscillatory recurre nt neural network (cornn): An accurate and (gradient) stable architecture fo r learning long time dependencies. arXiv preprint arXiv:2010.00951 . 32 Saneesh Cleatus Thundiyil, Joseph Picone, S.M.,
arXiv 2010
-
[15]
URL: https://arxiv.org/abs/ 1711.11053, arXiv:1711.11053
A multi- horizon quantile recurrent forecaster. URL: https://arxiv.org/abs/ 1711.11053, arXiv:1711.11053. Wu, H., Xu, J., Wang, J., Long, M.,
-
[16]
CAAI Transactions on Intelligence Technology 7, 129–143
A com- prehensive review on deep learning approaches in wind forec asting ap- plications. CAAI Transactions on Intelligence Technology 7, 129–143. doi:https://doi.org/10.1049/cit2.12076. Yu, F., Koltun, V., Funkhouser, T.,
-
[24]
URL: https://arxiv.org/abs/2310.06625, arXiv:2310.06625
itransformer: Inverted transformers are effective for time series forecasting. URL: https://arxiv.org/abs/2310.06625, arXiv:2310.06625. Makridakis, S., Spiliotis, E., Assimakopoulos, V.,
-
[32]
Deep independently r ecurrent neural network (indrnn)
Li, S., Li, W., Cook, C., Gao, Y., 2019b. Deep independently r ecurrent neural network (indrnn). arXiv preprint arXiv:1910.06251 . Li, Y., Yu, R., Shahabi, C., Liu, Y.,
arXiv 1910
-
[1988]
Journal of Complexity 4, 193–215
On the capabilities of multilayer percep- trons. Journal of Complexity 4, 193–215. URL: https://www. sciencedirect.com/science/article/pii/0885064X88900209, doi:https://doi.org/10.1016/0885-064X(88)90020-9 . Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.K., 1
Show all 18 references
-
[1992]
Dynamic network mo dels for forecasting, in: Uncertainty in artificial intelligence, E lsevier. pp. 41–48. Das, A., Kong, W., Leach, A., Mathur, S., Sen, R., Yu, R., 2024 . Long-term forecasting with tide: Time-series dense encoder. URL: https://arxiv. org/abs/2304.08424, arXiv...
2024 arXiv
-
[1998]
Journal of Business & Economic Statistics 16, 450–458
Cointegration a nd long-horizon forecasting. Journal of Business & Economic Statistics 16, 450–458. URL: http://www.jstor.org/stable/1392613. Chung, J., Gulcehre, C., Cho, K., Bengio, Y.,
-
[2014]
ar Xiv preprint arXiv:1412.3555
Empirica l evaluation of gated recurrent neural networks on sequence modeling. ar Xiv preprint arXiv:1412.3555 . Cirstea, R.G., Guo, C., Yang, B., Kieu, T., Dong, X., Pan, S.,
-
[2017]
CoRR abs/1702 .07805
Revisiting NA RX recurrent neural networks for long-term dependencies. CoRR abs/1702 .07805. URL: http://arxiv.org/abs/1702.07805, arXiv:1702.07805. Dudek, G., Pełka, P., Smyl, S.,
-
[2018]
arXiv preprint arX iv:1804.06776
Improving long-h orizon forecasts with expectation-biased lstm networks. arXiv preprint arX iv:1804.06776 . Kag, A., Zhang, Z., Saligrama, V.,
-
[2019]
arXiv preprint arXiv:1905.10437
N-beats: Neural basis expansion analysis for interpretable time series for ecasting. arXiv preprint arXiv:1905.10437 . Patton, A.J., Timmermann, A.,
1905 arXiv
-
[2020]
arXiv preprint arXiv:2001.04451
Reformer: The effi cient trans- former. arXiv preprint arXiv:2001.04451 . Komunjer, I., Owyang, M.T.,
2001 arXiv
-
[2021]
11106–11115
Informer: Beyond efficient transformer for long sequence tim e-series fore- casting, in: Proceedings of the AAAI conference on artificia l intelligence, pp. 11106–11115. Zhou, T., Ma, Z., wang, X., Wen, Q., Sun, L., Yao, T., Yin, W., J in, R., 2022a. Film: Frequency improved lege...
-
[2022]
a rXiv preprint arXiv:2211.14730
A time series is worth 64 words: Long-term forecasting with transformers. a rXiv preprint arXiv:2211.14730 . Oreshkin, B.N., Carpov, D., Chapados, N., Bengio, Y.,
-
[2024]
https://isip.piconepress.com/publications/book_sections/ 2025/springer/transformers/
Tran s- former architectures in time series analysis: A review (in p ublica- tion). https://isip.piconepress.com/publications/book_sections/ 2025/springer/transformers/. [Online; accessed 08-Nov-2024]. Semenoglou, A.A., Spiliotis, E., Assimakopoulos, V., 2023 . Data augmenta- ...
2025
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.