{"id":"79e2c142-9f9b-4b63-bce4-c27a80d4484d","arxiv_id":"2505.08550","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"OLinear forecasts time series by decorrelating them with the eigenbasis of their temporal correlation matrix, then applying linear layers; it reports strong accuracy across 24 benchmarks and high efficiency.","lead":"OLinear is a time-series forecasting model that rotates each series into a data-adaptive orthogonal basis, computed from the series' own correlation structure, before applying linear layers to predict future values. It also introduces NormLin, a lightweight linear layer with attention-like row-normalized weights, and reports consistent gains when it is swapped into transformer forecasters.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"OrthoTrans's decorrelation claim is not established on the actual RevIN-normalized inputs: the covariance identity in Section 4.2 is distributional and does not hold for the per-window inputs the model consumes, so the stated mechanism behind the plug-in gains in Tables 4-5 is unsupported.","rationale":"The reader's weakest assumption is that the decorrelation benefit requires the empirical training-set Pearson correlation to capture test-period dependence, which is exact only for Gaussian stationary data. I agree with that concern, but the sharper problem is more internal: even under stationarity, the actual model input is a per-window RevIN-normalized, dimension-extended vector, whose covariance is not the training-set CorrMat_t used to build Q_i. Section 4.2 slides from an eigenvalue identity for a hypothetical input with covariance CorrMat_t to a statement about the real pipeline. This makes the theoretical motivation for OrthoTrans weaker than the paper claims. The empirical substitution evidence in Tables 5, 9, and 10 is still the best part of the paper and is worth preserving; the issue is that the stated mechanism is not tested. The proposed check directly measures whether the mechanism operates on held-out data. Because the reader's verdict is already CONDITIONAL and this concern reinforces rather than overturns that condition, I recommend no change to the verdict. The overclaim is in the interpretation of the experiments, not in the core NormLin substitution results, which remain credible evidence for the module's usefulness.","tokens_in":94908,"tokens_out":6581,"duration_ms":74951,"concrete_test":"On held-out windows with the exact inference preprocessing, compute C = Cov(RevINNorm(X)) for each benchmark in Table 5, then compare the relative off-diagonal Frobenius energy of Q_i^T C Q_i, F^T C F (Fourier), and C (identity). If the OrthoTrans rotation does not reduce off-diagonal energy substantially below both Fourier and identity on PEMS03, ECL, Solar, and Weather, then OrthoTrans is not actually decorrelating the features the model consumes, and the plug-in gains in Tables 4 and 5 cannot be attributed to the stated mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism is stated in Section 4.2: after writing CorrMat_t = Q_i Lambda Q_i^T, the paper claims Cov(Q_i^T x) = Q_i^T CorrMat_t Q_i = Lambda, and concludes that 'the entries of Q_i^T x are linearly independent, removing sequential dependencies.' This is load-bearing because OrthoTrans is the paper's first contribution and the modular gains in Tables 4 and 5 are explained through this decorrelation argument. Two gaps undermine it. First, the identity is distributional, not per-instance. The model does not feed raw training windows into Q_i^T; it feeds RevIN-normalized windows, each recentered and rescaled, after an outer product with a learnable vector (Eq. 2 and Figure 1). The covariance of those actual inputs need not equal CorrMat_t, which is the average Pearson matrix of raw lagged training windows. So even in-sample, the rotation need not diagonalize the features the model actually sees. Second, even if the covariance were diagonal, that gives uncorrelatedness, not statistical independence; the paper's 'linearly independent' statement is false in general. The Gaussian conditional-mean theorem in Section 3 supplies the independence-style simplification only under joint Gaussianity. On non-stationary benchmarks such as Exchange, NASDAQ, SP500, DowJones, and COVID-19, train-period correlations are a weak proxy for test-period dependence. Thus the stated reason OrthoTrans 'consistently improves' existing forecasters is not established on the actual evaluation pipeline.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes OLinear, a linear multivariate time-series forecaster that operates in an orthogonally transformed domain. The first contribution, OrthoTrans, is a data-adaptive orthogonal transformation obtained from the eigendecomposition of the temporal Pearson correlation matrix of the training series; it is applied to RevIN-normalized inputs and is also proposed as a plug-in module for existing forecasters. The second contribution, NormLin, is a linear layer whose weight matrix is made positive via Softplus and row-wise L1-normalized, used as a cross-series learner and proposed as a drop-in replacement for self-attention in Transformer-based forecasters. The paper reports extensive experiments on 24 benchmarks and 140 forecasting tasks, including long- and short-term forecasting, few-shot and zero-shot evaluation, hyperparameter sensitivity, ablations, and efficiency measurements, and claims consistent state-of-the-art performance plus consistent plug-in improvements from both modules.","tokens_in":95053,"tokens_out":8880,"duration_ms":87930,"significance":"If the plug-in results hold, NormLin is a genuinely interesting finding: a simple normalized linear token-mixer matches or beats multi-head self-attention in the tested forecasting settings at roughly half the compute, and the controlled substitutions in Tables 5, 9, and 10 with unchanged hyperparameters are strong evidence in its favor. The paper also provides full result tables, seed-robustness statistics, FLOP/memory arithmetic that checks out, and an anonymous code repository, which are commendable for reproducibility. The main weakness is that the theoretical mechanism claimed for OrthoTrans is not established for the actual model inputs, and the unqualified 'state-of-the-art' and 'consistently improves' claims are stronger than the presented tables support.","major_comments":[{"comment":"The decorrelation argument for OrthoTrans is not established for the inputs the model actually consumes. The identity Cov(Q_i^T x) = Lambda presumes that x has covariance CorrMat_t and that Q_i^T is applied to x directly, but Eq. (2) applies Q_i^T to Z = (RevINNorm(X) ⊗ phi_d), which is generated from per-window instance-normalized and outer-product-scaled inputs; the covariance of such features need not equal CorrMat_t, so even in-sample the rotation need not diagonalize the features seen by the encoder. In addition, the statement in §4.2 that 'the entries of Q_i^T x are linearly independent, removing sequential dependencies' is not a valid consequence of a diagonal covariance: diagonal covariance gives uncorrelatedness, and statistical independence would require joint Gaussianity, which is not established for these benchmarks. Finally, the Gaussian conditional-expectation theorem in §3 does not by itself convert forecasting into an 'inter-independent feature prediction task,' because after transforming only x, the conditional mean still depends on all transformed coordinates through Sigma_xy^T Q_i. Since OrthoTrans is the paper's first contribution and its gains in Tables 4 and 5 are explained through this mechanism, please either verify empirically that the actual transformed features are near-diagonal on in-sample and held-out windows, or revise the stated mechanism and motivation to what is demonstrated (e.g., the rank-increasing and energy-compaction observations and the empirical plug-in gains).","section":"§4.2 and §3"},{"comment":"The claim that OLinear 'consistently achieves state-of-the-art performance' is not supported by the tables as presented. In Table 2, TimeMixer++ has lower average MSE than OLinear on ETT (0.349 vs. 0.359), Traffic (0.416 vs. 0.451), Weather (0.226 vs. 0.237), and Solar-Energy (0.203 vs. 0.215). In Table 3, iTransformer beats OLinear on Website setting S1 (0.179 vs. 0.225 MSE) and DLinear beats OLinear on Unemp (0.565 vs. 0.729 MSE). Similarly, the phrase 'consistently improves' for the plug-in modules is only true on averages: in Table 19, PatchTST with OrthoTrans on PEMS03 at horizon 96 worsens from 0.269 to 0.293 MSE, and in Table 10, Timer zero-shot on Traffic worsens from 0.458 to 0.462 MSE. Please report an aggregate statistic such as mean/median rank across the 140 tasks, or qualify the claims as average improvements rather than consistent per-dataset, per-horizon wins.","section":"Abstract and §5.1"}],"minor_comments":[{"comment":"The Wavelet1 row reports an ILI (S1) MSE of 0.116, which appears to be a typo for 1.116 as given in Table 18; please correct it.","section":"Table 4"},{"comment":"The model name is spelled 'OrthoLienar' in the headers of Tables 16 and 17; please fix the typo.","section":"Tables 16–17"},{"comment":"The entry for Weather H2 is written as '0.204±3-3'; this should presumably be '0.204±3e-3', and other entries should be checked for the same notation error.","section":"Table 24"},{"comment":"The sentence 'this property also ensures robust performance in few-shot learning scenarios (see Appendix ??)' contains an unresolved cross-reference; please replace it with the actual appendix or remove the pointer.","section":"Appendix I.8"},{"comment":"The text around Eq. (2) and the caption of Figure 2 contains encoding artifacts (sequences like '/uni00000013/uni00000018/...'); these should be cleaned before publication.","section":"Eq. (2) and Figure 2"},{"comment":"The word 'consistently' is used repeatedly for NormLin plug-in results (e.g., Abstract and §5.3), but the data in Table 10 show a zero-shot Traffic case with a small degradation (0.458 to 0.462 MSE); please state the claim as average behavior or explicitly note exceptions.","section":"§5.3"}],"recommendation":"major_revision","confidential_remarks":"The strongest part of the paper is the controlled plug-in evidence in Tables 5, 9, and 10, where hyperparameters are unchanged and the improvements are consistent at the average level. The weakest part is the theoretical justification of OrthoTrans in §3–§4.2, which needs either a corrected derivation or an empirical demonstration on the actual transformed inputs. The 'consistently state-of-the-art' phrasing should be tempered with aggregate statistics or explicit caveats. I would not reject the paper, but the current claims outrun the evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the controlled plug-in experiments are the real contribution and they hold up: swapping NormLin into four transformers and OrthoTrans into iTransformer/PatchTST/RLinear produces consistent gains with unchanged hyperparameters, and the efficiency numbers are right. Second, the stated mechanism for those gains is not established. Section 4.2 claims Q_i^T x has diagonal covariance because Cov(Q_i^T x) = Q_i^T CorrMat_t Q_i = Lambda, but that identity is about raw training windows. The model actually feeds RevIN-normalized, outer-product-expanded windows into Q_i^T, so the covariance of the actual features need not be diagonal even in-sample. And even if it were, diagonal covariance gives uncorrelatedness, not 'linear independence'—a false statement in general. The Gaussian conditional-mean theorem in Section 3 does not rescue this; it merely shows the conditional mean is linear for Gaussians, not that decorrelation makes features independent on non-stationary real data. So the theory oversells what the experiments show.\n\nWhat the paper does well: the plug-in framing is the right test. Tables 5, 9, 10 are the heart; they show consistent improvement without retuning. NormLin is simple and the Jacobian analysis in Appendix B is a reasonable attempt to explain why it may train better than softmax attention. The ablation breadth is impressive—24 benchmarks, 140 tasks—and the authors are honest in Appendix L about the O(N^2) scaling and forecasting-only scope.\n\nSoft spots, in proportion. The abstract's 'consistently achieves state-of-the-art' is contradicted by their own Table 2: TimeMixer++ wins on Traffic, Weather, Solar. That language should be tempered. The benchmark protocol mixes literature numbers with locally run baselines while OLinear gets per-dataset hyperparameter search; this asymmetry can flatter the new method, though the plug-in experiments are controlled and mitigate it. Horizon-averaged results shrink variance, which the authors concede.\n\nBottom line: the empirical contribution is real and the paper deserves a serious referee. The theory needs rewriting, not the experiments. I'd suggest the authors either weaken the decorrelation claim to what the data shows (consistent plug-in gains) or redo the covariance analysis on the actual transformed features.","headline":"Solid empirical plug-in results with an overstated theoretical narrative; the decorrelation mechanism doesn't actually apply to the RevIN-normalized inputs.","tokens_in":95802,"tokens_out":2722,"would_cite":true,"duration_ms":27265,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Rotating time series into their own correlation eigenbasis lets a linear model beat transformer forecasters.","keywords":["time series forecasting","orthogonal transformation","correlation matrix","decorrelation","linear model","self-attention","NormLin","multivariate forecasting"],"falsifier":"On a held-out series from a different regime than the training window (e.g., a financial index before and after a volatility shift), compute the average absolute off-diagonal entry of $\\mathrm{Cov}(Q_i^T x_{\\text{test}})$: if it stays large, the decorrelation premise fails and the accuracy gap over time-domain linear models should vanish. For the NormLin claim, rerun the controlled substitution of self-attention with NormLin in a transformer on a new high-variate dataset with all hyperparameters fixed; a failure to match or beat the vanilla MSE would refute the plug-in claim.","tokens_in":94512,"feed_emoji":"📈","tokens_out":7526,"duration_ms":69355,"temperature":0.7,"pith_summary":"OLinear is an attempt to show that the entangled step-to-step dependencies in raw multivariate time series are a solvable obstacle: rotate the data into a dataset-specific orthogonal basis before learning, and a plain linear model can forecast better than transformer-based rivals. The rotation, OrthoTrans, is built from the eigenvectors of the temporal Pearson correlation matrix of the training series, which makes the transformed coordinates linearly uncorrelated. The second component, NormLin, is a linear layer constrained to have positive, row-normalized weights, giving it the shape of an attention matrix without the query-key machinery. Across 24 benchmarks and 140 forecasting tasks, the authors report that OLinear matches or beats the comparison forecasters, and that both modules improve existing models when inserted as plug-ins, with NormLin using roughly half the FLOPs of multi-head self-attention.","feed_headline":"A linear forecaster beats attention after one rotation","feed_subtitle":"Rotating series onto their correlation eigenvectors lets a plain linear model match transformers at half the FLOPs.","key_machinery":"The load-bearing object is the pair (OrthoTrans, NormLin). OrthoTrans computes the averaged temporal Pearson correlation matrix $\\mathrm{CorrMat}_t$ from $T$ lagged copies of each variate, performs the eigenvalue decomposition $\\mathrm{CorrMat}_t=Q_i\\Lambda Q_i^T$, and multiplies the input by $Q_i^T$ so that $Q_i^T x$ has diagonal covariance; the output-side matrix $Q_o$ maps predictions back to the time domain. NormLin is the linear layer $x\\mapsto \\mathrm{RowNorm}_{L1}(\\mathrm{Softplus}(W))x$, whose positive row-normalized weight matrix mimics the row-stochastic attention matrix of self-attention. Together they carry the argument: OrthoTrans supplies a decorrelated input that a linear encoder can exploit, and NormLin supplies cross-variate dependency modeling that is cheaper, higher-rank, and better conditioned for gradients than softmax attention.","core_discovery":"On the paper's own terms, the discovery is that forecasting is easier in a coordinate system that diagonalizes the series' empirical temporal correlation, and that a normalized linear layer captures multivariate dependence at least as well as self-attention. For an input window $x$, the transformation satisfies $\\mathrm{Cov}(Q_i^T x)=\\Lambda$, so the coordinates entering the model are approximately independent; the paper motivates this with a Gaussian conditional-expectation theorem stating that $\\mu_{y|x}=\\mu_y+\\Sigma_{xy}^T\\Sigma_x^{-1}(x-\\mu_x)$, where the second term is the entanglement that OrthoTrans removes. NormLin replaces the attention weight computation with $\\mathrm{RowNorm}_{L1}(\\mathrm{Softplus}(W))x$, preserving attention's positive row-stochastic structure while keeping the weight matrix high-rank and improving gradient flow. The paper's falsifiable core is the controlled-substitution evidence: inserting OrthoTrans into iTransformer, PatchTST, and RLinear, and inserting NormLin into iTransformer, PatchTST, Leddam, Fredformer, and Timer, improves average MSE in nearly every reported setting.","pith_inferences":["A natural extension the paper does not test: if a positive row-stochastic interaction matrix is the active ingredient, NormLin could replace attention in other token-mixing settings such as graph or set learning.","A stress test implied but not run: recompute or shrink $Q_i$ when the test regime's correlation structure shifts, since the paper only varies the amount of training data used to build $Q$.","A hybrid not explored: use OrthoTrans as a data-driven refinement of Fourier or wavelet bases rather than as a replacement, capturing both dataset-specific and generic frequency structure.","A diagnostic to try: measure the off-diagonal mass of $\\mathrm{Cov}(Q_i^T x_{\\text{test}})$ as the forecast horizon grows; if decorrelation decays at long horizons, the gains over time-domain linear models should shrink."],"forward_implications":["If the plug-in results hold, any transformer forecaster can swap self-attention for NormLin and expect MSE improvements (6.7% on iTransformer, 10.3% on PatchTST in the reported averages) while cutting FLOPs roughly in half.","If OrthoTrans generalizes as a module, dataset-specific correlation structure becomes a reusable preprocessing step that improves iTransformer, PatchTST, and even the single-layer RLinear.","The success of OLinear-C, which fixes NormLin's weights to Softmax(CorrMat_v), implies that much of what the cross-series learner learns is the empirical correlation structure among variates; a cheaper fixed-weight variant nearly matches the learned one.","Because NormLin improved zero-shot and fine-tuning performance when substituted into the large decoder-only forecaster Timer, linear token-dependency layers may scale to pre-trained time-series foundation models.","The reported efficiency numbers mean state-of-the-art forecasting can be maintained with a fraction of the compute of transformer baselines."],"supporting_citations":[{"why":"Supplies the self-attention design whose positive row-stochastic structure NormLin imitates and replaces.","marker":"[Vaswani et al., 2017]"},{"why":"Motivates transformed-domain forecasting and the use of linear encoding on decorrelated features.","marker":"[Yi et al., 2023]"},{"why":"Provides the DLinear baseline and the linear-mapping perspective used in the Intra-Series Learner.","marker":"[Zeng et al., 2023]"},{"why":"Supplies RevIN instance normalization, used to mitigate non-stationarity before OrthoTrans is applied.","marker":"[Kim et al., 2021]"},{"why":"CARD supplies the weighted L1 loss adopted by OLinear and serves as a transformer baseline.","marker":"[Wang et al., 2024b]"},{"why":"Grounds the Pearson correlation matrix computation and decorrelation used by OrthoTrans.","marker":"[Gray and Davisson, 2004]"},{"why":"Provides the eigenvalue decomposition and block-matrix facts used in Theorem 1 and in building $Q_i$ and $Q_o$.","marker":"[Horn and Johnson, 2012]"},{"why":"Supplies the PCA rationale for energy compaction and noise suppression in the transformed domain.","marker":"[Jolliffe, 2002]"},{"why":"Timer is the large pre-trained forecaster used to test NormLin's scalability and zero-shot behavior.","marker":"[Liu et al., 2024b]"},{"why":"Supports the claim that higher-rank attention or weight matrices better preserve representation space.","marker":"[Han et al., 2023]"}],"fun_headline_variants":["After one rotation, linear beats attention","Rotate series, drop attention, keep linear","Diagonalize time to beat attention","Linear at half the FLOPs beats attention","One rotation makes linear outperform attention"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole benefit rests on the training-set Pearson correlation matrix being a faithful picture of the dependence that will be faced at test time, so that rotating onto its eigenvectors truly removes the temporal entanglement; real non-stationary series, including financial data, only satisfy this approximately.","fun_headline_variants_meta":{"raw":{"variants":["After one rotation, linear beats attention","Rotate series, drop attention, keep linear","Diagonalize time to beat attention","Linear at half the FLOPs beats attention","One rotation makes linear outperform attention"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001332,"raw_usage":{"total_tokens":5473,"prompt_tokens":1053,"completion_tokens":4420,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":669,"completion_tokens_details":{"reasoning_tokens":4357}},"tokens_in":669,"tokens_out":4420,"duration_ms":27695,"temperature":1.0,"reasoning_tokens":4357,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:19:32.357077+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a held-out series from a different regime than the training window (e.g., a financial index before and after a volatility shift), compute the average absolute off-diagonal entry of $\\mathrm{Cov}(Q_i^T x_{\\text{test}})$: if it stays large, the decorrelation premise fails and the accuracy gap over time-domain linear models should vanish. For the NormLin claim, rerun the controlled substitution of self-attention with NormLin in a transformer on a new high-variate dataset with all hyperparameters fixed; a failure to match or beat the vanilla MSE would refute the plug-in claim.","supporting_citations":[],"review_version":1}