Pith. sign in

REVIEW 5 major objections 7 minor 30 references

Temporal Window Smoothing of Exogenous Variables for Improved Time Series Prediction

T0 review · 5 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Projecting each exogenous window onto the top global principal components and reconstructing it gives transformer forecasts a better conditioning signal.

desk verdict TWS is feature-space PCA projection, not temporal smoothing; the central claim is unsupported by the equations and the paper needs major revision. read the letter →

arxiv 2507.05284 v1 pith:WYBYTDMW submitted 2025-07-04 cs.LG

Mustafa Kamal , Niyaz Bin Hashem , Robin Krambroeckers , Nabeel Mohammed , Shafin Rahman This is my paper
classification cs.LG
keywords timeseriesforecastingexogenousvariablesprincipalcomponentanalysistemporalwindowsmoothingtransformerwhiteningpreprocessingglobalstatistics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that forecasting models using exogenous variables can be improved by a parameter-free preprocessing step called Temporal Window Smoothing (TWS). Instead of feeding each window of exogenous series to the model in raw form, TWS projects the window onto the global principal components of the training data, keeps the components that explain at least 90% of variance, and reconstructs the window from them. This removes redundant information and exposes the exogenous input to patterns spanning the whole training set, without lengthening the lookback window. In experiments across seven benchmark datasets, the method beats 11 baselines in four datasets and reports the best MSE in 21 of 35 settings, with the main gains coming on ETT and Weather data. The interest of the claim is that better exogenous conditioning can be bought by a purely statistical preprocessing transform rather than a more complex model.

What carries the argument

The load-bearing object is the Temporal Window Smoothing (TWS) block: a global PCA basis $V$ and mean vector $\mu$ computed once from the training exogenous series, a dynamic rank selection that keeps the smallest $k$ with cumulative explained variance ratio at least 0.90, and the projection–reconstruction pair $\Psi = V^\top E$, $O = V\Psi + \mu$. This pair maps each window into the span of the top global components and back, which is what removes redundancy, suppresses components that are globally insignificant, and gives each window a view of patterns spanning the whole series. Because $k$ is chosen per dataset, the method adapts the amount of smoothing to the data's intrinsic dimensionality.

What would settle it

Train the same architecture on a dataset whose exogenous variables have one covariance structure in training and a different covariance structure in testing, for example a sine wave whose phase and frequency regime changes after the train/test split. If TWS does not improve over raw exogenous inputs on such a split, or hurts, the fixed global basis is the limiting assumption. A direct measurement is the reconstruction error $\|E - O\|$ on test windows: when it is large relative to training windows, TWS is projecting test data into a stale subspace.

Watch

Extended reading notes

Core claim

The central discovery is that whitening each window of exogenous series against a global PCA basis learned from the entire training set makes the exogenous input a better conditioning signal for transformer forecasting. For a window $E$, the TWS block computes $\Psi = V^\top E$ using the top-$k$ eigenvectors $V$ of the training covariance, with $k$ chosen as the smallest number of components whose cumulative explained variance reaches 90%, and then reconstructs $O = V\Psi + \mu$. The reconstructed series has the same dimensions as the original but lives in the subspace of globally significant patterns, so it is less redundant and more aware of long-term trends and seasonality. This refined input is then projected and integrated through a global token and cross-attention, and the whole pipeline is trained with MSE loss. The paper argues that the improvement over raw exogenous integration is due to this smoothing rather than to extra capacity, since TWS adds no learnable parameters.

Load-bearing premise

The weakest premise is that the PCA basis and mean computed from the training exogenous series continue to represent the exogenous series seen at test time; if the feature space or the distribution of the exogenous data shifts, the low-rank reconstruction can discard signal the model needed.

Editorial extensions

If this is right

  • TWS can be dropped into any transformer architecture that already accepts exogenous inputs without adding trainable parameters or increasing the lookback length.
  • The dynamic 90% variance rule makes the smoothing self-tuning across datasets with different numbers of exogenous channels.
  • On the four datasets where it leads, the method reports state-of-the-art averages; overall it records the best MSE in 21 of 35 forecast settings and the best MAE in 18 of 35.
  • The ablation attributes the gain to the combination of TWS with the global-token cross-attention bridge, since TWS alone improves that setup on ETTh1, ETTm1, and Weather.
  • If the raw exogenous input is highly redundant, TWS acts as a denoiser; if it is noisy, the reconstruction filters that noise before the model sees it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because TWS is a fixed preprocessing transform, it is likely to transfer to non-transformer backbones such as linear or MLP forecasters; the paper lists this as future work, and the mechanism itself does not depend on attention.
  • A testable extension is to replace the fixed training-time basis with a rolling or online PCA basis; for nonstationary exogenous series this may recover some of the signal that a globally fixed subspace discards.
  • The 90% variance criterion may be suboptimal when the exogenous signal relevant to the target is a small-variance component; one could test whether choosing $k$ by downstream forecast validation error rather than explained variance yields further gains.
  • The same projection–reconstruction idea could be applied to the endogenous input itself, although the paper restricts TWS to exogenous variables to keep the endogenous channel untouched.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes Temporal Window Smoothing (TWS), a non-learned preprocessing block for exogenous inputs in transformer-based multivariate time series forecasting. TWS is described as computing principal components from the global training set, projecting each exogenous window onto the top components that explain 90% of variance, and reconstructing a whitened exogenous input before it is integrated with endogenous patch embeddings via attention. Experiments are reported on ECL, Traffic, ETT, and Weather against 11 baselines, with the abstract claiming state-of-the-art results on four benchmark datasets and consistent outperformance of all baselines.

Significance. If the method worked exactly as described, it would be a simple, parameter-free preprocessing technique with broad applicability, and the paper provides a large benchmark table and an ablation study as evidence. Credit is due for releasing code and for including dense comparisons. However, the central mechanism as written is not temporal smoothing, and the empirical summary is contradicted by the paper's own Table III. These issues currently prevent the reader from assessing the actual contribution, so the significance is not yet established.

major comments (5)
  1. [Section III-B and Eq. (5)] The core operation is not temporal smoothing. With V in R^(N x k) and E in R^(N x Lex), Eq. (2) gives Psi = V^T E, so each column of Psi depends only on the corresponding column E_{:,t}; Eq. (3) then reconstructs O_{:,t} = V V^T E_{:,t} + mu. The covariance matrix Sigma is computed as (1/(Lseq-1)) * Xtilde * Xtilde^T, an N x N matrix that is invariant to permuting the time axis. Therefore the projection cannot encode trends, seasonality, or extended temporal context. The phrases 'temporal window smoothing' and 'aware of patterns and trends over extended periods' in Sections I and III-B are not derivable from the equations. The paper must either adopt a genuinely temporal PCA (e.g., eigenvectors of a Lex x Lex temporal covariance matrix) or thoroughly reframe the contribution as cross-sectional low-rank filtering of exogenous features.
  2. [Eq. (5) and Algorithm 1] The matrix dimensions are inconsistent. Algorithm 1 declares E in R^(N x Lex) and V in R^(N x k), but line 8 writes Psi = E_{1:Lex} x V, which is undefined. Line 10 then writes O = Psi V^T + mu; if Psi is k x Lex, this product is also undefined. Equation (5) repeats the same inconsistency. The only dimensionally coherent readings are either (a) E is transposed to Lex x N, in which case the result is the same column-wise feature-space projection already discussed, or (b) V is in R^(Lex x k), which would be a genuinely temporal PCA. The manuscript does not specify which reading is intended, so the exact algorithm is under-specified and the reader cannot determine what was actually implemented.
  3. [Abstract and Table III] The claim that the approach 'consistently outperforms 11 baseline models' is contradicted by Table III. On ECL, TimeXer has lower MSE than the proposed method at every horizon (e.g., 0.140 vs 0.148 at H=96); on Traffic, both TimeXer and iTransformer beat the proposed method at every horizon; on ETTh2, TimeXer is better at H=336 and H=720. Thus the method is state-of-the-art on at most four datasets, and even those statements need qualification because the improvements are small in several settings. The abstract and Section IV-B must be corrected to report these losses accurately and should not claim consistent dominance over all baselines.
  4. [Section III-B and Section IV-A] The construction of the exogenous variables E is never specified, which makes the experiments irreproducible. The method needs to state, for each benchmark dataset, which channels are treated as endogenous and which as exogenous, and how the PCA basis V and mean mu estimated from X_train are aligned with the feature space of E. Section III-B says the PCA is computed on 'the entire training data {X^i_train}_{i=1}^N, where N is the number of features', but it is not explained whether this training set is the endogenous series, the exogenous series, or the union. Without this alignment, a central premise of the method is unverified.
  5. [Table II] The ablation results do not support a uniform claim that TWS improves forecasting. With the Concat integration, TWS worsens ETTh1 (MSE 0.439 to 0.446) and Weather (MSE 0.255 to 0.261), improving only ETTm1. With Cross-Attention integration, TWS improves all three datasets, but the differences are small (e.g., ETTh1 MSE 0.445 to 0.430) and no error bars or significance tests are reported. The paper should either restrict the claim of benefit to the Cross-Attention architecture or discuss why TWS interacts negatively with concatenation.
minor comments (7)
  1. [Section III-B] The term 'whitening' is used loosely. Equations (2)-(3) describe a low-rank PCA projection and reconstruction, not whitening, which would involve scaling components to unit variance. The terminology should be aligned with the actual operation.
  2. [Eq. (3)] Equation (3) adds mu after reconstruction, but Equation (2) projects E without subtracting mu. If E is not mean-centered before projection, the reconstruction is not a valid inverse of the centering step. The equations should consistently show projection of (E - mu) and reconstruction as V V^T (E - mu) + mu.
  3. [Eq. (12)] Equation (12) reads bY = M_theta(X_{1:L}, TWS(O_{1:Lex})), but O is already the output of TWS. Applying TWS again would transform the smoothed input a second time. This is likely a typo and should be M_theta(X_{1:L}, O_{1:Lex}).
  4. [Section IV-B] The text says the comparison includes 'nine state-of-the-art deeplearning forecasting models', but Table III lists 11 baselines in addition to the proposed method. The counts should be reconciled.
  5. [Section III-B] The 'dynamic mechanism' for selecting k is a fixed cumulative-explained-variance threshold of 90%, chosen once per dataset, not dynamically per window. The paper should describe it as a fixed hyperparameter and ideally provide sensitivity analysis around the 0.90 threshold.
  6. [Figure 3 caption] The caption contains a duplicated word: 'to to obtain the final predictions'. This should be corrected.
  7. [Section IV-C] The ablation results are reported as averages over prediction lengths without variance or significance information; given the small differences, run-to-run variability could change the qualitative conclusion. Reporting multiple seeds would strengthen the claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the TWS preprocessing is estimated from exogenous training statistics and evaluated on external benchmarks; no prediction reduces to a fitted input.

full rationale

I searched the derivation chain for self-definitional definitions, fitted inputs renamed as predictions, or load-bearing self-citations. The PCA basis V and mean vector mu are estimated from the training exogenous series via eigendecomposition of the covariance matrix Sigma; the number of components k is chosen by cumulative explained variance (>= 0.90), not by forecast error or target values. The reconstructed exogenous window O is then fed into a transformer trained with MSE, and the reported gains are empirical benchmark comparisons against external baselines. No equation equates a claimed prediction to a fitted parameter by construction, and no central premise depends on the authors' prior work. The skeptical concern that Eq. (5) and Algorithm 1 are dimensionally inconsistent or that the method is per-column rather than temporal is a correctness/under-specification issue, not a circularity issue. Therefore the paper is not circular in the sense defined here.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

TWS introduces no new physical or architectural entities; it is a preprocessing transform. The main hand-chosen or data-fitted quantities are the 90% variance threshold and the resulting component count k. The method also rests on stationarity and feature-alignment assumptions that are not explicitly justified.

free parameters (2)
  • Variance threshold for principal component selection = 0.90
    Section III-B selects k as the smallest value with cumulative explained variance >= 0.90. The threshold is a hand-chosen constant that controls how many components are retained.
  • Number of principal components k = Not reported per dataset
    k is data-dependent and derived from the 0.90 threshold, but the paper never reports the k values used. This makes the exact preprocessing hard to reproduce without running PCA independently.
assumptions (3)
  • domain assumption PCA basis V and mean mu estimated on the training split remain valid for test windows
    Section III-B and Eq. (2)-(3) apply training-derived V and mu to every window; a distribution shift or regime change would make the reconstruction discard information.
  • ad hoc to paper Exogenous features E and the features used to estimate V live in the same N-dimensional space
    Eq. (2) requires V in R^{N x k} and E in R^{N x L_ex}, but V is described as derived from X_train, with no stated alignment between X_train and E. This is an unstated assumption needed for the method to be well-defined.
  • domain assumption The top-k PCA components preserve forecasting signal and the discarded components are redundant or noisy
    The denoising argument assumes that low-variance directions carry little predictive value; the paper provides no direct evidence for this beyond the reported benchmark numbers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Temporal Window Smoothing of Exogenous Variables for Improved Time Series Prediction." pith.science (2026). https://pith.science/paper/WYBYTDMW

@misc{pith2026250705284,
  author       = {Pith},
  title        = {Pith review of: Temporal Window Smoothing of Exogenous Variables for Improved Time Series Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WYBYTDMW}},
  note         = {Machine review of arXiv:2507.05284}
}
read the original abstract

Although most transformer-based time series forecasting models primarily depend on endogenous inputs, recent state-of-the-art approaches have significantly improved performance by incorporating external information through exogenous inputs. However, these methods face challenges, such as redundancy when endogenous and exogenous inputs originate from the same source and limited ability to capture long-term dependencies due to fixed look-back windows. In this paper, we propose a method that whitens the exogenous input to reduce redundancy that may persist within the data based on global statistics. Additionally, our approach helps the exogenous input to be more aware of patterns and trends over extended periods. By introducing this refined, globally context-aware exogenous input to the endogenous input without increasing the lookback window length, our approach guides the model towards improved forecasting. Our approach achieves state-of-the-art performance in four benchmark datasets, consistently outperforming 11 baseline models. These results establish our method as a robust and effective alternative for using exogenous inputs in time series forecasting.

Figures

Figures reproduced from arXiv: 2507.05284 by the authors.

Figure 1
Figure 1. Visualization of our proposed approach for time series forecasting: [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Block diagram of (a-b) traditional approaches where raw exogenous [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overall architecture of the proposed method consists of three main components. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

30 extracted references · 21 canonical work pages

  1. [1]

    Interpretable weather forecasting for worldwide stations with a unified deep model,

    H. Wu, H. Zhou, M. Long, and J. Wang, “Interpretable weather forecasting for worldwide stations with a unified deep model,” Nature Machine Intelligence, 2023

    work page 2023

  2. [2]

    Skilful nowcasting of extreme precipitation with nowcastnet,

    Y . Zhang, M. Long, K. Chen, L. Xing, R. Jin, M. I. Jordan, and J. Wang, “Skilful nowcasting of extreme precipitation with nowcastnet,” Nature, 2023

    work page 2023

  3. [3]

    Forecasting day- ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark,

    J. Lago, G. Marcjasz, B. De Schutter, and R. Weron, “Forecasting day- ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark,” Applied Energy, vol. 293, p. 116983, 2021

    work page 2021

  4. [4]

    Electricity price forecasting: A review of the state-of-the-art with a look into the future,

    R. Weron, “Electricity price forecasting: A review of the state-of-the-art with a look into the future,” International journal of forecasting , 2014

    work page 2014

  5. [5]

    Financial time series forecasting using cnn and transformer,

    Z. Zeng, R. Kaur, S. Siddagangappa, S. Rahimi, T. Balch, and M. Veloso, “Financial time series forecasting using cnn and transformer,” arXiv preprint arXiv:2304.04912, 2023

    arXiv 2023

  6. [6]

    Traffic flow prediction with big data: A deep learning approach,

    Y . Lv, Y . Duan, W. Kang, Z. Li, and F.-Y . Wang, “Traffic flow prediction with big data: A deep learning approach,” IEEE Transactions on Intelligent Transportation Systems , vol. 16, no. 2, pp. 865–873, 2014

    work page 2014

  7. [7]

    Comparison of sarimax, sarima, modified sarima and ann-based models for short-term pv generation forecasting,

    S. I. Vagropoulos, G. Chouliaras, E. G. Kardakos, C. K. Simoglou, and A. G. Bakirtzis, “Comparison of sarimax, sarima, modified sarima and ann-based models for short-term pv generation forecasting,” in ENERGYCON, 2016

    work page 2016

  8. [8]

    Timexer: Empowering transformers for time series forecasting with exogenous variables,

    Y . Wang, H. Wu, J. Dong, Y . Liu, Y . Qiu, H. Zhang, J. Wang, and M. Long, “Timexer: Empowering transformers for time series forecasting with exogenous variables,” Advances in Neural Information Processing Systems, 2024

    work page 2024

Show all 30 references
  1. [9]

    Multivariate vehicular traffic flow prediction: evalua- tion of arimax modeling,

    B. M. Williams, “Multivariate vehicular traffic flow prediction: evalua- tion of arimax modeling,” Transportation Research Record , 2001

    2001

  2. [10]

    Long-term forecasting with tide: Time-series dense encoder,

    A. Das, W. Kong, A. Leach, R. Sen, and R. Yu, “Long-term forecasting with tide: Time-series dense encoder,” arXiv preprint arXiv:2304.08424, 2023

    2023 arXiv

  3. [11]

    itrans- former: Inverted transformers are effective for time series forecasting,

    Y . Liu, T. Hu, H. Zhang, H. Wu, S. Wang, L. Ma, and M. Long, “itrans- former: Inverted transformers are effective for time series forecasting,” arXiv preprint arXiv:2310.06625 , 2023

    2023 arXiv

  4. [12]

    Informer: Beyond efficient transformer for long sequence time-series forecasting,

    H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond efficient transformer for long sequence time-series forecasting,” in AAAI, 2021

    2021

  5. [13]

    En- hancing the locality and breaking the memory bottleneck of transformer on time series forecasting,

    S. Li, X. Jin, Y . Xuan, X. Zhou, W. Chen, Y .-X. Wang, and X. Yan, “En- hancing the locality and breaking the memory bottleneck of transformer on time series forecasting,” in NeurIPS, 2019

    2019

  6. [14]

    Timesnet: Temporal 2d-variation modeling for general time series analysis,

    H. Wu, T. Hu, Y . Liu, H. Zhou, J. Wang, and M. Long, “Timesnet: Temporal 2d-variation modeling for general time series analysis,” ICLR, 2023

    2023

  7. [15]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  8. [16]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020

    2010 arXiv

  9. [17]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in ICCV, 2021

    2021

  10. [18]

    Autoformer: Decomposition transformers with Auto-Correlation for long-term series forecasting,

    H. Wu, J. Xu, J. Wang, and M. Long, “Autoformer: Decomposition transformers with Auto-Correlation for long-term series forecasting,” NeurIPS, 2021

    2021

  11. [19]

    Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting,

    S. Liu, H. Yu, C. Liao, J. Li, W. Lin, A. X. Liu, and S. Dustdar, “Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting,” in ICLR, 2022

    2022

  12. [20]

    Fedformer: Frequency enhanced decomposed transformer for long-term series fore- casting,

    T. Zhou, Z. Ma, Q. Wen, X. Wang, L. Sun, and R. Jin, “Fedformer: Frequency enhanced decomposed transformer for long-term series fore- casting,” in ICML, 2022

    2022

  13. [21]

    Simmtm: A simple pre-training framework for masked time-series modeling,

    J. Dong, H. Wu, H. Zhang, L. Zhang, J. Wang, and M. Long, “Simmtm: A simple pre-training framework for masked time-series modeling,” in NeurIPS, 2023

    2023

  14. [22]

    A time series is worth 64 words: Long-term forecasting with transformers,

    Y . Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam, “A time series is worth 64 words: Long-term forecasting with transformers,” ICLR, 2023

    2023

  15. [23]

    Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with nbeatsx,

    K. G. Olivares, C. Challu, G. Marcjasz, R. Weron, and A. Dubrawski, “Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with nbeatsx,” International Journal of F orecasting , 2023

    2023

  16. [24]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems, 2017

    2017

  17. [25]

    Revisiting long-term time se- ries forecasting: An investigation on linear mapping,

    Z. Li, S. Qi, Y . Li, and Z. Xu, “Revisiting long-term time se- ries forecasting: An investigation on linear mapping,” arXiv preprint arXiv:2305.10721, 2023

    2023 arXiv

  18. [26]

    Crossformer: Transformer utilizing cross- dimension dependency for multivariate time series forecasting,

    Y . Zhang and J. Yan, “Crossformer: Transformer utilizing cross- dimension dependency for multivariate time series forecasting,” in ICLR, 2022

    2022

  19. [27]

    Are transformers effective for time series forecasting?

    A. Zeng, M. Chen, L. Zhang, and Q. Xu, “Are transformers effective for time series forecasting?” AAAI, 2023

    2023

  20. [28]

    Scinet: time series modeling and forecasting with sample convolution and interaction,

    M. Liu, A. Zeng, M. Chen, Z. Xu, Q. Lai, L. Ma, and Q. Xu, “Scinet: time series modeling and forecasting with sample convolution and interaction,” NeurIPS, 2022

    2022

  21. [29]

    Koopa: Learning non-stationary time series dynamics with koopman predictors,

    Y . Liu, C. Li, J. Wang, and M. Long, “Koopa: Learning non-stationary time series dynamics with koopman predictors,” in NeurIPS, 2023

    2023

  22. [30]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 , 2014. APPENDIX This section provides the detailed results of the ablation study shown in Table. II. The best results are marked in red . TABLE IV FULL ABLATION STUDY ON LONG ...

    2014 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.