{"id":"0f93ffc8-86c1-418d-8b01-076d7660abdf","arxiv_id":"2506.12809","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A survey of long-horizon forecasting with new ETTm2 ablations showing per-timestep error growth that is absent for xLSTM and Triformer.","lead":"This paper reviews 35 years of long-horizon time series forecasting research and adds new benchmark results on the ETTm2 dataset. It reports that errors grow with forecast horizon for most models, except xLSTM and Triformer, and frames long-horizon forecasting as an error propagation problem.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The xLSTM exception in the per-timestep heatmap is confounded by the paper's nonstandard strided-convolution implementation, so the central 'error propagation' claim is not yet supported.","rationale":"The reader's weakest assumption, test-set hyperparameter selection, is real and is supported by the paper's own wording in Section 8.1. However, I found a more specific and more damaging confound for the central claim: the xLSTM variant used in the paper is nonstandard, with a strided convolution and repeated features, and the authors explicitly say the stride is visible in the multivariate heatmap. That makes the per-timestep error shape for xLSTM uninterpretable as evidence about recurrent memory or error propagation. The paper also does not provide the per-timestep values numerically; Table 4 contains only one aggregate MSE per model per setting, so the 'steady increase except xLSTM/Triformer' claim cannot be independently checked from the manuscript. My proposed check, comparing vanilla xLSTM and a strided-convolution-only baseline, would settle whether the exception actually survives. Because the reader's verdict is already CONDITIONAL and my concern reinforces the need for empirical clarification rather than moving the verdict, I mark verdict_should_be as UNCHANGED.","tokens_in":21320,"tokens_out":5109,"duration_ms":48930,"concrete_test":"Re-run the Table 4 multivariate per-timestep experiment with the same hyperparameter selection protocol but with a vanilla xLSTM (no strided convolution, no repeated features) and, separately, with a non-recurrent MLP baseline that uses the same strided-convolution/repeated-feature input pipeline. If the flat or non-monotonic per-timestep MSE pattern persists in the MLP baseline or disappears in vanilla xLSTM, the claimed xLSTM exception is an artifact of the strided convolution rather than of xLSTM memory. Additionally, report the per-timestep MSE curves numerically with a fitted slope and bootstrap confidence intervals over multiple seeds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that per-timestep MSE rises steadily over the horizon for most architectures but not for xLSTM and Triformer, motivating LHF as an error propagation problem. The xLSTM exception is not attributable to xLSTM's recurrent memory because the authors implemented xLSTM with a strided convolution 'of length equal to the kernel size and repeated features to maintain the length' (Section 7.4), and Section 8.1 states that the 'strided convolution-based implementation is visible in its multivariate forecasting heatmap... corresponding to the stride value.' A stride-induced periodic feature-repetition pattern can easily masquerade as a flat or non-monotonic per-step error curve. The same section also says the reversal appears 'when the models find the best hyperparameters over the ETTm2 test set,' indicating test-set-driven configuration search. The paper does not publish the per-timestep curves numerically, only aggregate H=720 MSE values in Table 4, and no error bars or seeds are given. Therefore the exception claim is currently a visually interpreted confound rather than a supported finding, and the 'error propagation' motivation rests on a single dataset and an unvalidated xLSTM variant.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper is a review of long-horizon forecasting (LHF) in time series analysis, covering statistical and deep-learning approaches over roughly the last 35 years. It surveys decomposition techniques, windowing schemes, preprocessing methods, and model families (MLPs, RNNs, self-attention, and pyramidal self-attention), and it reports ablation experiments on the ETTm2 dataset in both multivariate and univariate (HUFL) settings for horizons H = 96, 192, 336, 720. The authors use a nonstandard 16/4/4 month train/validation/test split, compare their results to those in the cited literature, and present per-timestep MSE heatmaps. The central empirical claim, stated in the abstract, is that per-timestep MSE increases steadily with horizon length for most models except xLSTM and Triformer, which the authors interpret as motivating LHF as an error-propagation problem.","tokens_in":21521,"tokens_out":4126,"duration_ms":39768,"significance":"If the central claim were properly supported, the paper would make a useful contribution: it consolidates a large literature, provides a consistent ablation across several model families on a common benchmark, and proposes a concrete mechanistic interpretation of long-horizon degradation. The public release of trained models and the systematic comparison with cited results (including the 7:1:2 split delta in Table 1) are commendable and increase the reproducibility of the survey portion. However, the load-bearing empirical claim is currently supported only by visual inspection of heatmaps from single runs, with no numerical per-timestep curves, no uncertainty quantification, a hyperparameter-selection protocol that may involve test-set feedback, and an xLSTM variant whose strided-convolution implementation is acknowledged by the authors to be visible in the heatmap. These issues make the paper's main interpretation premature.","major_comments":[{"comment":"The central claim of the paper, that per-timestep MSE increases steadily with horizon for all models except xLSTM and Triformer, rests entirely on visual inspection of the heatmaps in Table 4. The per-timestep curves are not published numerically, no error bars or repeated seeds are reported, and no statistical test is applied to the monotonicity or to the proposed exceptions. The abstract states this pattern as a finding, but the evidence is a single-run visual pattern. Please report the numerical per-step MSE curves averaged over multiple seeds with confidence intervals, and quantify the monotonic increase and the exceptions (e.g., slope estimates or a contrast test) so that the claim is falsifiable.","section":"§8.1, Table 4"},{"comment":"The hyperparameter configurations are described in Section 7.4 as those 'that worked well with respect to the Mean Average Error (MAE),' with no validation protocol stated, and Section 8.1 says the xLSTM reversal appears 'when the models find the best hyperparameters over the ETTm2 test set.' If test-set performance was used to select configurations, the reported comparisons and the xLSTM/Triformer exception are confounded by test-set feedback. Please state the exact selection protocol and, if the test set was used, rerun the experiments under a strict train/validation/test protocol and confirm that the heatmap patterns persist.","section":"§7.4, §8.1"},{"comment":"The xLSTM exception cannot currently be attributed to xLSTM's recurrent memory: the implementation uses a strided convolution 'of length equal to the kernel size and repeated features to maintain the length' (Section 7.4), and Section 8.1 states that this implementation 'is visible in its multivariate forecasting heatmap... corresponding to the stride value.' A stride-induced periodic feature-repetition pattern can produce a non-monotonic per-step error curve that is unrelated to error propagation or memory. Please run the canonical xLSTM (or an ablation without the strided convolution) and show that the flat or non-monotonic pattern persists.","section":"§7.4, §8.1"},{"comment":"The empirical support for the 'error propagation' framing is based on a single dataset (ETTm2) with a nonstandard 16/4/4 split (Section 7.2). Even if the heatmap patterns were confirmed, the claim that LHF is 'an error propagation problem' as a general characterization would require evidence across datasets and horizons. As written, the paper offers an interpretation of one dataset's heatmaps. Please either add additional datasets or explicitly restrict the claim to ETTm2 and describe it as a hypothesis rather than a general finding.","section":"§7.2, §8.1"}],"minor_comments":[{"comment":"The Δ column uses nonstandard symbols (✓, ×, downward arrows) that are only partially explained in the caption; please make the notation fully self-contained, including what the downward arrow means for the entries where no prior score is cited.","section":"Table 1"},{"comment":"The heatmaps are not reproducible from the text: the table reports only aggregate MSE values with embedded images, and the underlying per-timestep arrays are not provided. Please make the numerical per-step error curves available, for example in the model repository or in an appendix.","section":"Table 4"},{"comment":"In Equation (10), the stabilization state mt and the exponential gates are introduced, but the superscript convention for the stabilized input gate (i^s_t) is not defined before its first use, which makes the equation hard to parse for a reader not already familiar with the xLSTM paper.","section":"§6.2.3"},{"comment":"The sentence on State Space Models contains a subject–verb disagreement ('has number of parameters'), and the referenced SpaceTime results are mentioned without a direct comparison in the ablation tables; please either add the comparison or clearly mark it as external.","section":"§8.3"},{"comment":"The phrase 'over the last 35 years or so' is informal for a journal abstract, and the paper uses 'this review' to refer to both the present article and the literature it covers in several places; please use 'this paper' for the present work to avoid ambiguity.","section":"Abstract and §1"}],"recommendation":"major_revision","confidential_remarks":"The review portion is broad and likely useful to practitioners, but the empirical contribution is thin relative to the strength of the abstract's claim. The main risk is that the xLSTM 'exception' is an artifact of the nonstandard strided-convolution implementation and possibly of test-set-based hyperparameter selection; this needs to be resolved before the error-propagation framing can be accepted. I also note that the paper would benefit from a clearer statement of what is new beyond the survey: as written, the ablations on a single dataset with a modified split may not meet the empirical standards of a forecasting journal. There are no concerns about attribution or citation practice beyond the usual need to cite the original xLSTM and Triformer implementations more precisely."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read. The paper is a broad survey of long-horizon forecasting (LHF) with a small empirical ablation on ETTm2. The survey is genuinely useful: it organizes the MLP, RNN, self-attention, and pyramidal-attention families, provides equations, and discusses decomposition, windowing, normalization, and patching in a way that would help a new entrant. Shipping pretrained weights is a plus. That part deserves credit.\n\nThe empirical centerpiece—per-timestep MSE grows with horizon except for xLSTM and Triformer, motivating LHF as an error-propagation problem—does not survive a close read. The observation is not new: the direct-versus-recursive forecasting literature already covers error accumulation in iterative multi-step forecasting, and the authors themselves cite Zeng et al. on this point. More importantly, the exceptions are confounded. xLSTM is implemented with a strided convolution 'of length equal to the kernel size and repeated features to maintain the length,' and Section 8.1 says the strided pattern is visible in the heatmap. So the flat/non-monotonic error curve for xLSTM may be an artifact of the convolution stride, not of recurrent memory. Triformer's 'exception' is based on visual inspection of heatmaps, with no statistical test. The paper also says the xLSTM pattern appears 'when the models find the best hyperparameters over the ETTm2 test set'—test-set-driven configuration search. No seeds, no error bars, one dataset, and the per-step curves are not published numerically.\n\nThere are smaller issues. The split is nonstandard (6:2:2 instead of the commonly used 12/4/4 or 7:1:2), which the authors acknowledge but it still makes cross-paper comparisons shaky. Some model groups are mentioned but not covered in the ablations, e.g., SSMs. None of these are fatal for a review; they are fatal for the empirical claim.\n\nNet: I would not cite this for the error-propagation conclusion. I would cite it, if at all, for the survey and the model zoo. Who gets value? Students and practitioners wanting a map of model families and preprocessing tricks. Should it go to peer review? Yes, as a major-revision candidate: the empirical section needs a stated validation protocol, seeds/error bars, and either a standard split or a clear justification for the new one; the xLSTM variant needs to be described precisely enough to rule out the stride confound; and the claims should be scaled back to what the data support. The survey alone justifies a referee's time.","headline":"A useful survey of long-horizon forecasting that overreaches on its empirical error-propagation claim, which is confounded by a nonstandard xLSTM variant and test-set-driven hyperparameter selection.","tokens_in":22035,"tokens_out":4405,"would_cite":false,"duration_ms":38301,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Forecast errors grow with horizon length—except in two architectures","keywords":["long horizon forecasting","time series forecasting","error propagation","self-attention models","xLSTM","Triformer","ETTm2","ablation study"],"falsifier":"Re-run the same ablation study on ETTm2 with all hyperparameters chosen using only the validation split, then recompute the per-timestep MSE heatmaps on the untouched test set; if xLSTM and Triformer still show non-growing error while the other models grow, the error-propagation claim is supported, and if not, the exceptional behavior was an artifact of configuration search.","tokens_in":21084,"feed_emoji":"📈","tokens_out":6185,"duration_ms":52011,"temperature":0.7,"pith_summary":"This review argues that long-horizon forecasting (LHF) fails because error accumulates across the forecast horizon, and that most neural architectures exhibit this accumulation. It surveys the field across MLP, RNN, self-attention, and pyramidal self-attention models, and reports ablations on the ETTm2 dataset over horizons of 96 to 720 steps. The paper's central empirical observation is a set of heatmaps showing average per-step MSE rising in proportion to horizon length for nearly all models, with xLSTM and Triformer as the exceptions. If correct, the paper reframes LHF from a problem of capturing long-range dependence to a problem of suppressing error propagation over the horizon.","feed_headline":"Forecast errors grow with horizon length—except in two architectures","feed_subtitle":"On ETTm2, per-step error rises across nearly all models; xLSTM and Triformer stay flat.","key_machinery":"The central object is the per-timestep MSE heatmap: for each test series, the model's squared error at every position of the forecast horizon is averaged across the test set, producing a curve of error versus horizon distance. For most models that curve rises with horizon length; for xLSTM and Triformer it stays comparatively flat. The argument works by contrasting these curves across model families and attributing the flat exceptions to architectural mechanisms: Triformer's factorized variable-specific attention over triangular patches, and xLSTM's strided-convolution input processing with scalar and matrix recurrent memory. The heatmap is what turns the otherwise standard benchmark scores into a claim about error propagation.","core_discovery":"The paper claims that long-horizon forecasting should be viewed as an error propagation problem. On ETTm2, the authors compute, for each time step along the forecast horizon, the MSE averaged over the test set, and find that error rises steadily with distance for the MLP, self-attention, and pyramidal self-attention models under study. The exceptions are xLSTM and Triformer, whose per-step errors do not grow with horizon length. Triformer's hierarchical triangular attention distributes the forecast across scales, while xLSTM's scalar and matrix memory states with exponential gating appear to reverse the dependence on length. The paper motivates this pattern as evidence that LHF is governed by error accumulation, and offers the heatmap representation as the diagnostic that exposes it.","pith_inferences":["The error-propagation claim rests on one dataset (ETTm2); a straightforward extension would be to recompute these per-step heatmaps on electricity, traffic, and weather benchmarks to see whether the rising-error curve is a general law or an ETTm2 artifact.","Because the hyperparameters were tuned on MAE without a stated validation protocol, the flat heatmaps of xLSTM and Triformer need a robustness check: repeat the search with validation-based early stopping and observe whether the flat pattern survives.","The paper's own note that NHITS also shows a similar flat pattern in the multivariate heatmap hints that the real mechanism is multi-resolution pooling rather than recurrence per se, which would broaden the design space beyond the two named exceptions."],"forward_implications":["If error propagation is the binding constraint, then horizon-length error curves, not aggregate MSE alone, should be the primary diagnostic for LHF models.","Architectures that escape the rising-error pattern—hierarchical attention and recurrent memory with stabilized gating—become the natural templates for designing longer-horizon models.","The survey supports the earlier finding that transformer-style attention does not automatically solve LHF on this benchmark, since the rising-error pattern appears across the self-attention family too.","A model that cannot keep per-step error flat will continue to degrade as the horizon grows, even if its aggregate score is competitive at short horizons."],"supporting_citations":[{"why":"Supplies the ETTm2 dataset and the Informer baseline model used in the ablations.","marker":"Zhou et al. (2021)"},{"why":"Defines Triformer, the hierarchical triangular-attention model that shows flat per-step error.","marker":"Cirstea et al. (2022)"},{"why":"Defines NBEATS, a core MLP baseline for the ablation comparisons.","marker":"Oreshkin et al. (2019)"},{"why":"Defines NHITS, the pooling/interpolation MLP baseline and LHF-focused method.","marker":"Challu et al. (2023)"},{"why":"Defines Autoformer, a self-attention baseline in the comparison.","marker":"Wu et al. (2021)"},{"why":"Defines FEDformer, another self-attention baseline using frequency transforms.","marker":"Zhou et al. (2022b)"},{"why":"Defines Pyraformer, the pyramidal self-attention baseline.","marker":"Liu et al. (2021)"},{"why":"Defines D-Linear and N-Linear, the linear/MLP baselines central to the LHF discussion.","marker":"Zeng et al. (2023)"}],"fun_headline_variants":["MSE climbs along forecast horizon—xLSTM and Triformer defy it","Why long-horizon errors grow—and which models escape","Error propagation rules long-horizon forecasting, save two","Two models break the error-growth trend in long horizons","Horizon length drives forecast error—two models resist"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported comparisons and the exceptional flat-error behavior of xLSTM and Triformer assume that the hyperparameter search, which selected configurations for good MAE without a described validation protocol, did not overfit the test set.","fun_headline_variants_meta":{"raw":{"variants":["MSE climbs along forecast horizon—xLSTM and Triformer defy it","Why long-horizon errors grow—and which models escape","Error propagation rules long-horizon forecasting, save two","Two models break the error-growth trend in long horizons","Horizon length drives forecast error—two models resist"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000356,"raw_usage":{"total_tokens":1926,"prompt_tokens":936,"completion_tokens":990,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":919}},"tokens_in":552,"tokens_out":990,"duration_ms":8651,"temperature":1.0,"reasoning_tokens":919,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:06:50.283974+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same ablation study on ETTm2 with all hyperparameters chosen using only the validation split, then recompute the per-timestep MSE heatmaps on the untouched test set; if xLSTM and Triformer still show non-growing error while the other models grow, the error-propagation claim is supported, and if not, the exceptional behavior was an artifact of configuration search.","supporting_citations":[],"review_version":1}