{"id":"27662163-f5d2-4374-b133-2d177fa4a937","arxiv_id":"2507.05284","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"PCA-based reconstruction of exogenous inputs, with a dynamic 90% variance component count, improves transformer forecasting MSE on ETTm1, ETTm2, ETTh1, and Weather, but not on ECL, ETTh2, or Traffic.","lead":"Temporal Window Smoothing (TWS) uses PCA to clean up extra data before it goes into a time series forecaster. The cleaned data improved results on some benchmarks, but the effect was small and did not appear on three of the seven datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The method as written is a per-time-step feature-space projection, not temporal smoothing; Eq. (5) and Algorithm 1 are dimensionally inconsistent, so the claimed global temporal context is unsupported.","rationale":"The reader's weakest assumption focused on feature-space alignment and distribution shift. I agree that alignment is a real risk, but the more load-bearing problem is internal: the only dimensionally consistent derivation in Section III-B defines an operation that is invariant to time ordering and therefore cannot provide the 'temporal window smoothing' or 'long-term trends and seasonality' awareness claimed in the abstract. Meanwhile Eq. (5) and Algorithm 1 contradict that derivation by using a matrix product that is undefined under the stated shapes. This means the paper does not currently specify a single algorithm: either the intended method is the per-time-step feature PCA of Eqs. (2)-(3), in which case the temporal-smoothing narrative is unsupported, or the intended method is a temporal PCA, in which case the covariance derivation and the definition of V are wrong. The empirical tables could still be valid for one of these variants, but the central claim cannot be evaluated from the manuscript alone. The reader's CONDITIONAL verdict is reasonable in spirit, but the condition should be stronger: the authors must reconcile the equations and confirm which operation the released code actually implements. Until then, the claim is unverdictable from the paper itself.","tokens_in":15730,"tokens_out":11660,"duration_ms":142482,"concrete_test":"Inspect the released code at github.com/robot-bulls/TWINS-E and report the shape of V and the multiplication order in the TWS block. If V is N x k and the forward pass computes V^T @ E (or E @ V with E shaped Lex x N), then derive the Jacobian dO_t/dE_{t'} for t' != t; for the paper's Eqs. (2)-(3) this Jacobian is exactly zero. If the Jacobian is nonzero, the implementation includes temporal operations not described. Additionally, rerun ETTh1/ECL with a random permutation of the time axis of the exogenous training block before PCA; because Sigma is a sum over time steps, V, mu, and O are unchanged, so any change in test MSE would indicate undocumented temporal processing.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"Section III-B defines V in R^(N x k) from the contemporaneous covariance Sigma = (1/(Lseq-1)) * Xtilde_train * Xtilde_train^T and reconstructs O = V V^T (E - mu) + mu (Eqs. 2-3). This reconstruction is column-wise: each O_{:,t} depends only on E_{:,t}, and Sigma is invariant to permuting the time axis. Therefore TWS cannot encode trends, seasonality, or any temporal-window context; it is cross-sectional low-rank filtering across exogenous variables. The central claim that the exogenous input becomes 'aware of patterns and trends over extended periods' is not derivable from the equations. Independently, Eq. (5) and Algorithm 1 (lines 8-10) write Psi = E_{1:Lex} x V with V in R^(N x k) and E in R^(N x Lex); this matrix product is undefined. The only dimensionally consistent readings are: (a) E is transposed to Lex x N, yielding the same feature-space projection, or (b) V is actually in R^(Lex x k), which would be a genuinely temporal PCA. The paper does not specify which, so the actual algorithm is under-specified and the central claim is untestable from the manuscript.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Temporal Window Smoothing (TWS), a non-learned preprocessing block for exogenous inputs in transformer-based multivariate time series forecasting. TWS is described as computing principal components from the global training set, projecting each exogenous window onto the top components that explain 90% of variance, and reconstructing a whitened exogenous input before it is integrated with endogenous patch embeddings via attention. Experiments are reported on ECL, Traffic, ETT, and Weather against 11 baselines, with the abstract claiming state-of-the-art results on four benchmark datasets and consistent outperformance of all baselines.","tokens_in":15998,"tokens_out":7710,"duration_ms":87875,"significance":"If the method worked exactly as described, it would be a simple, parameter-free preprocessing technique with broad applicability, and the paper provides a large benchmark table and an ablation study as evidence. Credit is due for releasing code and for including dense comparisons. However, the central mechanism as written is not temporal smoothing, and the empirical summary is contradicted by the paper's own Table III. These issues currently prevent the reader from assessing the actual contribution, so the significance is not yet established.","major_comments":[{"comment":"The core operation is not temporal smoothing. With V in R^(N x k) and E in R^(N x Lex), Eq. (2) gives Psi = V^T E, so each column of Psi depends only on the corresponding column E_{:,t}; Eq. (3) then reconstructs O_{:,t} = V V^T E_{:,t} + mu. The covariance matrix Sigma is computed as (1/(Lseq-1)) * Xtilde * Xtilde^T, an N x N matrix that is invariant to permuting the time axis. Therefore the projection cannot encode trends, seasonality, or extended temporal context. The phrases 'temporal window smoothing' and 'aware of patterns and trends over extended periods' in Sections I and III-B are not derivable from the equations. The paper must either adopt a genuinely temporal PCA (e.g., eigenvectors of a Lex x Lex temporal covariance matrix) or thoroughly reframe the contribution as cross-sectional low-rank filtering of exogenous features.","section":"Section III-B and Eq. (5)"},{"comment":"The matrix dimensions are inconsistent. Algorithm 1 declares E in R^(N x Lex) and V in R^(N x k), but line 8 writes Psi = E_{1:Lex} x V, which is undefined. Line 10 then writes O = Psi V^T + mu; if Psi is k x Lex, this product is also undefined. Equation (5) repeats the same inconsistency. The only dimensionally coherent readings are either (a) E is transposed to Lex x N, in which case the result is the same column-wise feature-space projection already discussed, or (b) V is in R^(Lex x k), which would be a genuinely temporal PCA. The manuscript does not specify which reading is intended, so the exact algorithm is under-specified and the reader cannot determine what was actually implemented.","section":"Eq. (5) and Algorithm 1"},{"comment":"The claim that the approach 'consistently outperforms 11 baseline models' is contradicted by Table III. On ECL, TimeXer has lower MSE than the proposed method at every horizon (e.g., 0.140 vs 0.148 at H=96); on Traffic, both TimeXer and iTransformer beat the proposed method at every horizon; on ETTh2, TimeXer is better at H=336 and H=720. Thus the method is state-of-the-art on at most four datasets, and even those statements need qualification because the improvements are small in several settings. The abstract and Section IV-B must be corrected to report these losses accurately and should not claim consistent dominance over all baselines.","section":"Abstract and Table III"},{"comment":"The construction of the exogenous variables E is never specified, which makes the experiments irreproducible. The method needs to state, for each benchmark dataset, which channels are treated as endogenous and which as exogenous, and how the PCA basis V and mean mu estimated from X_train are aligned with the feature space of E. Section III-B says the PCA is computed on 'the entire training data {X^i_train}_{i=1}^N, where N is the number of features', but it is not explained whether this training set is the endogenous series, the exogenous series, or the union. Without this alignment, a central premise of the method is unverified.","section":"Section III-B and Section IV-A"},{"comment":"The ablation results do not support a uniform claim that TWS improves forecasting. With the Concat integration, TWS worsens ETTh1 (MSE 0.439 to 0.446) and Weather (MSE 0.255 to 0.261), improving only ETTm1. With Cross-Attention integration, TWS improves all three datasets, but the differences are small (e.g., ETTh1 MSE 0.445 to 0.430) and no error bars or significance tests are reported. The paper should either restrict the claim of benefit to the Cross-Attention architecture or discuss why TWS interacts negatively with concatenation.","section":"Table II"}],"minor_comments":[{"comment":"The term 'whitening' is used loosely. Equations (2)-(3) describe a low-rank PCA projection and reconstruction, not whitening, which would involve scaling components to unit variance. The terminology should be aligned with the actual operation.","section":"Section III-B"},{"comment":"Equation (3) adds mu after reconstruction, but Equation (2) projects E without subtracting mu. If E is not mean-centered before projection, the reconstruction is not a valid inverse of the centering step. The equations should consistently show projection of (E - mu) and reconstruction as V V^T (E - mu) + mu.","section":"Eq. (3)"},{"comment":"Equation (12) reads bY = M_theta(X_{1:L}, TWS(O_{1:Lex})), but O is already the output of TWS. Applying TWS again would transform the smoothed input a second time. This is likely a typo and should be M_theta(X_{1:L}, O_{1:Lex}).","section":"Eq. (12)"},{"comment":"The text says the comparison includes 'nine state-of-the-art deeplearning forecasting models', but Table III lists 11 baselines in addition to the proposed method. The counts should be reconciled.","section":"Section IV-B"},{"comment":"The 'dynamic mechanism' for selecting k is a fixed cumulative-explained-variance threshold of 90%, chosen once per dataset, not dynamically per window. The paper should describe it as a fixed hyperparameter and ideally provide sensitivity analysis around the 0.90 threshold.","section":"Section III-B"},{"comment":"The caption contains a duplicated word: 'to to obtain the final predictions'. This should be corrected.","section":"Figure 3 caption"},{"comment":"The ablation results are reported as averages over prediction lengths without variance or significance information; given the small differences, run-to-run variability could change the qualitative conclusion. Reporting multiple seeds would strengthen the claims.","section":"Section IV-C"}],"recommendation":"major_revision","confidential_remarks":"The paper contains a salvageable preprocessing idea, but the central description is internally inconsistent and the advertised empirical claims are not supported by the reported table. A major revision is needed to specify the exact algorithm, correct the narrative of temporal smoothing, and revisit the benchmarking claims before the work can be evaluated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my take on 2507.05284. The central claim is that TWS gives exogenous inputs \"global context\" by exposing them to patterns over extended periods. That claim does not follow from the math. The operation is a per-time-step projection: each column of the exogenous window is projected onto a feature-space PCA basis and reconstructed. The covariance is computed from the training sequence across time, so the eigenvectors capture contemporaneous correlations between exogenous variables—not trends, seasonality, or anything temporal. TWS, as written, is cross-sectional low-rank filtering, not temporal window smoothing.\n\nCredit where it's due: the idea of preprocessing exogenous inputs with PCA before feeding them to a transformer is simple, parameter-free, and cheap. The authors set up the standard benchmark and include ablations with and without the block, which is the right instinct. Code availability helps reproducibility.\n\nThe problems are load-bearing. First, Eq. (5) and Algorithm 1 are dimensionally inconsistent: multiplying E (N x Lex) by V (N x k) is undefined unless Lex = N. The only consistent reading is Psi = V^T E, which still leaves the temporal claim unsupported. Second, the empirical story is oversold. Table III shows losses to TimeXer on ECL, ETTh2, and Traffic, and the ablation (Table II) shows TWS hurting performance on ETTh1 and Weather under concatenation. The \"state-of-the-art in four datasets\" is a selective reading of a couple of wins. No error bars, and the dataset table is sloppy, though those are minor compared to the mechanism problem.\n\nThe paper is not publishable as is. If the authors reframed it as denoising exogenous features via PCA and ran a focused study, there might be a modest contribution. As written, the gap between the narrative and the equations is too wide for a serious venue.\n\nI would still send it to peer review—it touches a live topic, the experiments are simple to check, and a referee could quickly pin down the flaw. But my expectation would be rejection or major rewrite. I would not cite it in its current form.","headline":"TWS is feature-space PCA projection, not temporal smoothing; the central claim is unsupported by the equations and the paper needs major revision.","tokens_in":16522,"tokens_out":4438,"would_cite":false,"duration_ms":50553,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Projecting each exogenous window onto the top global principal components and reconstructing it gives transformer forecasts a better conditioning signal.","keywords":["time series forecasting","exogenous variables","principal component analysis","temporal window smoothing","transformer","whitening","preprocessing","global statistics"],"falsifier":"Train the same architecture on a dataset whose exogenous variables have one covariance structure in training and a different covariance structure in testing, for example a sine wave whose phase and frequency regime changes after the train/test split. If TWS does not improve over raw exogenous inputs on such a split, or hurts, the fixed global basis is the limiting assumption. A direct measurement is the reconstruction error $\\|E - O\\|$ on test windows: when it is large relative to training windows, TWS is projecting test data into a stale subspace.","tokens_in":15530,"feed_emoji":"📈","tokens_out":5728,"duration_ms":61121,"temperature":0.7,"pith_summary":"The paper claims that forecasting models using exogenous variables can be improved by a parameter-free preprocessing step called Temporal Window Smoothing (TWS). Instead of feeding each window of exogenous series to the model in raw form, TWS projects the window onto the global principal components of the training data, keeps the components that explain at least 90% of variance, and reconstructs the window from them. This removes redundant information and exposes the exogenous input to patterns spanning the whole training set, without lengthening the lookback window. In experiments across seven benchmark datasets, the method beats 11 baselines in four datasets and reports the best MSE in 21 of 35 settings, with the main gains coming on ETT and Weather data. The interest of the claim is that better exogenous conditioning can be bought by a purely statistical preprocessing transform rather than a more complex model.","feed_headline":"PCA-smoothed exogenous inputs beat 11 baselines in forecasting","feed_subtitle":"Reconstructing each exogenous window from top global components cuts redundancy and lifts accuracy with no extra parameters.","key_machinery":"The load-bearing object is the Temporal Window Smoothing (TWS) block: a global PCA basis $V$ and mean vector $\\mu$ computed once from the training exogenous series, a dynamic rank selection that keeps the smallest $k$ with cumulative explained variance ratio at least 0.90, and the projection–reconstruction pair $\\Psi = V^\\top E$, $O = V\\Psi + \\mu$. This pair maps each window into the span of the top global components and back, which is what removes redundancy, suppresses components that are globally insignificant, and gives each window a view of patterns spanning the whole series. Because $k$ is chosen per dataset, the method adapts the amount of smoothing to the data's intrinsic dimensionality.","core_discovery":"The central discovery is that whitening each window of exogenous series against a global PCA basis learned from the entire training set makes the exogenous input a better conditioning signal for transformer forecasting. For a window $E$, the TWS block computes $\\Psi = V^\\top E$ using the top-$k$ eigenvectors $V$ of the training covariance, with $k$ chosen as the smallest number of components whose cumulative explained variance reaches 90%, and then reconstructs $O = V\\Psi + \\mu$. The reconstructed series has the same dimensions as the original but lives in the subspace of globally significant patterns, so it is less redundant and more aware of long-term trends and seasonality. This refined input is then projected and integrated through a global token and cross-attention, and the whole pipeline is trained with MSE loss. The paper argues that the improvement over raw exogenous integration is due to this smoothing rather than to extra capacity, since TWS adds no learnable parameters.","pith_inferences":["Because TWS is a fixed preprocessing transform, it is likely to transfer to non-transformer backbones such as linear or MLP forecasters; the paper lists this as future work, and the mechanism itself does not depend on attention.","A testable extension is to replace the fixed training-time basis with a rolling or online PCA basis; for nonstationary exogenous series this may recover some of the signal that a globally fixed subspace discards.","The 90% variance criterion may be suboptimal when the exogenous signal relevant to the target is a small-variance component; one could test whether choosing $k$ by downstream forecast validation error rather than explained variance yields further gains.","The same projection–reconstruction idea could be applied to the endogenous input itself, although the paper restricts TWS to exogenous variables to keep the endogenous channel untouched."],"forward_implications":["TWS can be dropped into any transformer architecture that already accepts exogenous inputs without adding trainable parameters or increasing the lookback length.","The dynamic 90% variance rule makes the smoothing self-tuning across datasets with different numbers of exogenous channels.","On the four datasets where it leads, the method reports state-of-the-art averages; overall it records the best MSE in 21 of 35 forecast settings and the best MAE in 18 of 35.","The ablation attributes the gain to the combination of TWS with the global-token cross-attention bridge, since TWS alone improves that setup on ETTh1, ETTm1, and Weather.","If the raw exogenous input is highly redundant, TWS acts as a denoiser; if it is noisy, the reconstruction filters that noise before the model sees it."],"supporting_citations":[{"why":"Supplies the exogenous-variable transformer integration design and the strongest baseline that TWS extends.","marker":"[8]"},{"why":"Provides the concatenation-style exogenous integration approach that the paper contrasts with its smoothing method.","marker":"[10]"},{"why":"A leading multivariate transformer baseline that TWS outperforms in six of seven datasets.","marker":"[11]"},{"why":"Supplies the ETT datasets and the efficient-transformer baseline used in the comparisons.","marker":"[12]"},{"why":"Source of the Electricity (ECL) benchmark used in the experiments.","marker":"[13]"},{"why":"Supplies the Traffic dataset and the train/validation/test splitting procedure adopted for fair comparison.","marker":"[14]"},{"why":"Provides the channel-independent patching representation that the endogenous stream is built on.","marker":"[22]"},{"why":"Represents the linear exogenous-based forecast model compared as a baseline.","marker":"[23]"}],"fun_headline_variants":["PCA-smoothed exogenous inputs beat 11 forecast baselines","Global PCA whitening cuts redundancy and lifts forecast accuracy","TWS: no-parameter PCA smoothing improves time series forecasts","Whitening exogenous windows with global PCA beats 11 baselines","Forecast better by smoothing exogenous inputs with global PCA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The weakest premise is that the PCA basis and mean computed from the training exogenous series continue to represent the exogenous series seen at test time; if the feature space or the distribution of the exogenous data shifts, the low-rank reconstruction can discard signal the model needed.","fun_headline_variants_meta":{"raw":{"variants":["PCA-smoothed exogenous inputs beat 11 forecast baselines","Global PCA whitening cuts redundancy and lifts forecast accuracy","TWS: no-parameter PCA smoothing improves time series forecasts","Whitening exogenous windows with global PCA beats 11 baselines","Forecast better by smoothing exogenous inputs with global PCA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000196,"raw_usage":{"total_tokens":1333,"prompt_tokens":893,"completion_tokens":440,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":358}},"tokens_in":509,"tokens_out":440,"duration_ms":4968,"temperature":1.0,"reasoning_tokens":358,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:04:35.830803+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same architecture on a dataset whose exogenous variables have one covariance structure in training and a different covariance structure in testing, for example a sine wave whose phase and frequency regime changes after the train/test split. If TWS does not improve over raw exogenous inputs on such a split, or hurts, the fixed global basis is the limiting assumption. A direct measurement is the reconstruction error $\\|E - O\\|$ on test windows: when it is large relative to training windows, TWS is projecting test data into a stale subspace.","supporting_citations":[{"cited_title":"Timexer: Empowering transformers for time series forecasting with exogenous variables,","cited_arxiv_id":null,"evidence_quote":"Supplies the exogenous-variable transformer integration design and the strongest baseline that TWS extends."},{"cited_title":"Informer: Beyond efficient transformer for long sequence time-series forecasting,","cited_arxiv_id":null,"evidence_quote":"Supplies the ETT datasets and the efficient-transformer baseline used in the comparisons."},{"cited_title":"En- hancing the locality and breaking the memory bottleneck of transformer on time series forecasting,","cited_arxiv_id":null,"evidence_quote":"Source of the Electricity (ECL) benchmark used in the experiments."},{"cited_title":"Timesnet: Temporal 2d-variation modeling for general time series analysis,","cited_arxiv_id":null,"evidence_quote":"Supplies the Traffic dataset and the train/validation/test splitting procedure adopted for fair comparison."},{"cited_title":"A time series is worth 64 words: Long-term forecasting with transformers,","cited_arxiv_id":null,"evidence_quote":"Provides the channel-independent patching representation that the endogenous stream is built on."},{"cited_title":"Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with nbeatsx,","cited_arxiv_id":null,"evidence_quote":"Represents the linear exogenous-based forecast model compared as a baseline."}],"review_version":1}