REVIEW 4 major objections 7 minor 44 references
Unlocking the Potential of Linear Networks for Irregular Multivariate Time Series Forecasting
T0 review · 4 major / 7 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read Adaptive linear networks beat prior state-of-the-art on irregular multivariate time series forecasting, cutting error by 11% and runtime by 52%.
desk verdict AiT is a sensible combination but the core mechanism is not new and the headline margins are built on unverified baseline numbers. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is ALinear, an adaptive linear layer that replaces the fixed weight matrix of a standard linear layer with a dynamically computed matrix. Given input time points and output time points, two small MLP embedders produce keys and queries, and the weight matrix is computed as softmax of the query–key dot product, followed by a weighted sum of the input. This yields a weight matrix whose shape and values adjust to the actual observation and query times, resolving the shape mismatch and numerical misalignment that static weights suffer on irregular data. When time information is absent or constant, learnable default matrices substitute for the embeddings, which lets ALinear reduce to an ordinary linear layer on regular time series.
What would settle it
Run AiT and the strongest baselines (especially T-PatchGNN) on the same four datasets under identical preprocessing, splits, and query definitions, and check whether AiT still achieves roughly 11% lower average MSE and 52% lower runtime than the reproduced baselines.
Extended reading notes
Core claim
The paper's central claim is that the key barrier to using linear networks on irregular time series is the static weight matrix, which cannot align with series of variable length and irregular timing. AiT addresses this with ALinear, an adaptive linear layer that generates its weights dynamically from the input and output time points via dot-product attention and softmax normalization. The temporal encoder uses ALinear to map each variable's irregular observations to a fixed-length embedding, and the predictor uses ALinear again to map that embedding to values at arbitrary future query times. Inter-variable asynchrony is handled by running a transformer over these fixed-size variable embeddings. The paper reports state-of-the-art accuracy and efficiency across four real-world datasets: PhysioNet, MIMIC, Activity, and USHCN.
Load-bearing premise
The comparison assumes that the baseline numbers not reproduced by the authors, which were taken from a prior paper, were produced under exactly the same data splits, query definitions, preprocessing, and evaluation protocol used for AiT.
Editorial extensions
If this is right
- If the reported results hold, AiT provides a computationally cheap linear-architecture alternative to neural-ODE and graph-based methods for irregular multivariate forecasting, with roughly half the runtime of the previous best model.
- Because ALinear reduces to a standard linear layer when time points are fixed, the same model can handle both regular and irregular time series without architectural changes.
- The approach removes the need for numerical ODE integration and patch-based alignment when modeling irregular samples, potentially simplifying deployment in healthcare and climate monitoring settings.
- The success of a linear network with time-adaptive weights suggests that avoiding static weight misalignment matters more than complex nonlinear dynamics for many IMTS forecasting tasks.
Reading between the lines
- A natural extension the authors do not pursue: applying ALinear as a drop-in replacement for linear layers in other architectures (such as recurrent or state-space models) to make them irregular-data-ready.
- The paper's ablation shows the static variable embedding is critical on high-missing-rate datasets, which suggests that a learned per-variable prior complements the adaptive temporal layer; this could be tested further on datasets with extreme missingness.
- Because the claimed accuracy gain is 11% averaged over four datasets but varies per dataset, a robustness check across more irregularity patterns (e.g., varying missing rates or sampling jitter) would clarify where the method's advantage is largest.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes AiT, a model for irregular multivariate time series (IMTS) forecasting that replaces the static linear layers of iTransformer with an Adaptive Linear (ALinear) module. ALinear computes a softmax weight matrix from embeddings of input and output time points, enabling variable-length and irregularly sampled series to be handled within a linear-network framework, and a Transformer-based spatial encoder captures variable correlations in a shared latent space. The paper evaluates AiT on PhysioNet, MIMIC, Activity, and USHCN against 20 baselines, reporting an 11% average accuracy improvement and a 52% average runtime reduction relative to prior state of the art, together with ablations, hyperparameter sensitivity, varying-horizon experiments, and an RMTS generality study.
Significance. If the empirical claims are substantiated, AiT is a practically valuable IMTS forecaster that extends the linear-network paradigm to irregular sampling. The paper's strengths are a deliberately simple architecture, a broad baseline suite, ablations that isolate each component, hyperparameter and horizon analyses, and an honest Appendix E limitation statement. However, the headline accuracy and runtime margins are conditional on the comparability of baseline numbers taken from T-PatchGNN, and no statistical significance testing is reported. The potential significance is real but currently unverified; the central claim needs additional empirical support.
major comments (4)
- [Section 4.1.2, Table 1, Appendix B.2] The central claim of an 11% average accuracy improvement rests on baseline numbers that were not reproduced by the authors. Table 1's footnote states that results for 18 of 20 baselines are 'taken from T-PatchGNN,' and Appendix B.2 says the authors 'primarily rely on the results presented in T-PatchGNN.' If the data splits, query definitions, irregular-series preprocessing, metric computation, or hardware used in T-PatchGNN differ from those used for AiT, both headline percentages (11% accuracy, 52% runtime) are not established. Please reproduce T-PatchGNN and the strongest baselines under the exact protocol used for AiT, or release code and the exact data-partition and query definitions so that protocol equivalence can be verified.
- [Section 4.2, Table 1; Appendix C.3] The paper reports mean and standard deviation over five seeds but no statistical significance tests. In particular, Appendix C.3 asserts that the RMTS differences are 'not a statistically significant difference' without describing any test. Because the main comparisons involve overlapping distributions (e.g., T-PatchGNN 4.98±0.08 vs. AiT 4.58±0.06 on PhysioNet MSE), the claimed superiority requires a paired significance test or at least confidence intervals or effect sizes. Please add such tests for the main tables and for the Appendix C.3 claim.
- [Table 2] The runtime comparison in Table 2 inherits the same protocol-equivalence problem: the baseline times are copied from T-PatchGNN and were presumably measured on different hardware or with different software configurations. The 52% average runtime reduction in the abstract depends on the comparability of these measurements. Please either reproduce the baselines in the same environment or state explicitly the hardware, software versions, and measurement protocol for every entry.
- [Abstract and Table 1 'Impr.' row] The headline '11% improvement' is an unweighted average of the four per-dataset Impr. values in Table 1, and each of those values appears to be an average of the MSE and MAE improvements, but this aggregation is not defined anywhere in the paper. Because the per-metric improvements are on different scales, the unweighted average is not a well-defined quantity. Please define the aggregation formula or report per-metric, per-dataset results only.
minor comments (7)
- [Appendix A, Eq. (9)] The claimed equivalence between ALinear and a static linear layer in Eq. (9) is only schematic: it omits the softmax nonlinearity and the low-rank factorization, so the '≈' should be presented as an intuition rather than a derivation.
- [Section 3.1 and Section 3.5, Eq. (6)] The symbol L denotes both the number of Transformer layers in Eq. (6) and the number of observations L_n in Section 3.1; please use different symbols to avoid ambiguity.
- [Table 5] The column header 'MA×10−2' in Table 5 is a typo; it should read 'MAE×10−2.'
- [Figure 2] The label 'TemporalEncoderConcatProjection' in Figure 2 lacks a separator and is difficult to read; please use a clearer label.
- [Appendix B.3] The statement that code will be released 'following the release' is circular; please specify the planned release mechanism and timeline, and provide the code repository or a clear availability statement in the manuscript.
- [Section 3.1 and Section 4.1.3] The problem definition refers to 'queries' q_n^j but does not specify how these query time points are generated from the future observation windows; please define their construction explicitly, since this is essential for reproducing the comparison.
- [Appendix E] Appendix E honestly lists limitations (limited domain coverage and stationarity dependence), but the abstract does not acknowledge these constraints; please temper the abstract's broad claim of superiority or move the limitations into the main text.
Circularity Check
No circular derivation: AiT's predictions come from a trained network evaluated on held-out splits; borrowed T-PatchGNN baseline numbers are external evidence, not fitted inputs or self-citation.
full rationale
The paper's central claim is empirical: AiT is trained by minimizing the MSE loss in Eq. (8) on training data and evaluated on held-out test splits, so the reported errors are not derivable from the model definition or from any fitted constant. ALinear (Eqs. 2-3) is an architectural component, and no equation equates the claimed 11% accuracy gain or 52% runtime gain to an input or fitted parameter. The main weakness is that 18 of 20 baseline results are 'taken from T-PatchGNN' (Table 1 note), and Appendix B.2 states that the authors 'primarily rely on the results presented in T-PatchGNN'; if the original protocols or hardware differed, the headline margins would be unverified. That is a protocol-equivalence and reproducibility gap, not a circular derivation, because T-PatchGNN is an external prior work by a different group and its numbers are measurements rather than premises that define AiT's outputs. Self-citations (Ref. [28] for RMTS datasets and Ref. [30] for related work) are auxiliary and not load-bearing. The paper's own limitations section narrows the generality claims but does not fold the conclusion back into the inputs. No step in the derivation chain reduces a prediction to its input by construction.
Assumptions & free parameters
free parameters (3)
- hidden_dim =
64
- num_heads =
4
- num_layers =
3
assumptions (3)
- domain assumption The evaluation protocol (data splits, query definitions, irregular-series preprocessing, metrics) used to generate the copied baseline results in Tables 1 and 2 is identical to the protocol used for AiT.
- domain assumption Softmax normalization over variable-length input observations yields representations with consistent output expectation across varying input lengths.
- domain assumption The learned static variable embedding h_stat can compensate for variables with no observations at all.
invented entities (2)
-
Learnable static variable representation h_stat
-
Default key and query matrices K_default and Q_default
Cite this review
Pith. "Pith review of Unlocking the Potential of Linear Networks for Irregular Multivariate Time Series Forecasting." pith.science (2026). https://pith.science/paper/WZOM4WHF
@misc{pith2026250500590,
author = {Pith},
title = {Pith review of: Unlocking the Potential of Linear Networks for Irregular Multivariate Time Series Forecasting},
year = {2026},
howpublished = {\url{https://pith.science/paper/WZOM4WHF}},
note = {Machine review of arXiv:2505.00590}
}
read the original abstract
Time series forecasting holds significant importance across various industries, including finance, transportation, energy, healthcare, and climate. Despite the widespread use of linear networks due to their low computational cost and effectiveness in modeling temporal dependencies, most existing research has concentrated on regularly sampled and fully observed multivariate time series. However, in practice, we frequently encounter irregular multivariate time series characterized by variable sampling intervals and missing values. The inherent intra-series inconsistency and inter-series asynchrony in such data hinder effective modeling and forecasting with traditional linear networks relying on static weights. To tackle these challenges, this paper introduces a novel model named AiT. AiT utilizes an adaptive linear network capable of dynamically adjusting weights according to observation time points to address intra-series inconsistency, thereby enhancing the accuracy of temporal dependencies modeling. Furthermore, by incorporating the Transformer module on variable semantics embeddings, AiT efficiently captures variable correlations, avoiding the challenge of inter-series asynchrony. Comprehensive experiments across four benchmark datasets demonstrate the superiority of AiT, improving prediction accuracy by 11% and decreasing runtime by 52% compared to existing state-of-the-art methods.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[25]
Satya Narayan Shukla and Benjamin M. Marlin. Multi-time attention networks for irregularly sampled time series. In International Conference on Learning Representations, 2021
work page 2021
-
[1]
Adebiyi Ariyo Ariyo, Aderemi Oluyinka Adewumi, and Charles K. Ayo. Stock price prediction using the ARIMA model. In International Conference on Computer Modelling and Simulation, 2014
work page 2014
-
[2]
Neural flows: Efficient alternative to neural odes
Marin Bilos, Johanna Sommer, Syama Sundar Rangapuram, Tim Januschowski, and Stephan Günnemann. Neural flows: Efficient alternative to neural odes. In Annual Conference on Neural Information Processing Systems, 2021
work page 2021
-
[3]
G. E. P. Box and G. M. Jenkins. Some recent advances in forecasting and control. Journal of the Royal Statistical Society, 17, 1968
work page 1968
-
[4]
John S. Bridle. Training stochastic model recognition algorithms as networks can lead to maximum mutual information estimation of parameters. In Annual Conference on Neural Information Processing Systems, 1989
work page 1989
-
[5]
Gru-ode-bayes: Continuous modeling of sporadically-observed time series
Edward De Brouwer, Jaak Simm, Adam Arany, and Yves Moreau. Gru-ode-bayes: Continuous modeling of sporadically-observed time series. In Annual Conference on Neural Information Processing Systems, 2019
work page 2019
-
[6]
Spectral temporal graph neural network for multivariate time-series forecasting
Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, and Qi Zhang. Spectral temporal graph neural network for multivariate time-series forecasting. In Annual Conference on Neural Information Processing Systems, 2020
work page 2020
-
[7]
Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David A. Sontag, and Yan Liu. Re- current neural networks for multivariate time series with missing values. arXiv, 1606.01865, 2016
arXiv 2016
Show all 44 references
-
[8]
Neural ordinary differential equations
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. In Annual Conference on Neural Information Processing Systems, 2018
2018
-
[9]
Yuzhou Chen, Ignacio Segovia-Dominguez, Baris Coskunuzer, and Yulia R. Gel. Tamp- s2gcnets: Coupling time-aware multipersistence knowledge representation with spatio-supra graph convolutional networks for time-series forecasting. In International Conference on Learning Represe...
2022
-
[10]
Deepar: Probabilistic forecasting with autoregressive recurrent networks
Valentin Flunkert, David Salinas, and Jan Gasthaus. Deepar: Probabilistic forecasting with autoregressive recurrent networks. arXiv, 2201.00382, 2017
2017 arXiv
-
[11]
Deep sparse rectifier neural networks
Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. In International Conference on Artificial Intelligence and Statistics, 2011
2011
-
[12]
Borgwardt
Max Horn, Michael Moor, Christian Bock, Bastian Rieck, and Karsten M. Borgwardt. Set functions for time series. In International Conference on Machine Learning, 2020
2020
-
[13]
Hu, elong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen
Edward J. Hu, elong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022
2022
-
[14]
Crossgnn: Confronting noisy multivariate time series via cross interaction refine- ment
Qihe Huang, Lei Shen, Ruixin Zhang, Shouhong Ding, Binwu Wang, Zhengyang Zhou, and Yang Wang. Crossgnn: Confronting noisy multivariate time series via cross interaction refine- ment. In Annual Conference on Neural Information Processing Systems, 2023
2023
-
[15]
Pickett, and Varun Dutt
Shruti Kaushik, Abhinav Choudhury, Pankaj Kumar Sheron, Nataraj Dasgupta, Sayee Natarajan, Larry A. Pickett, and Varun Dutt. AI in healthcare: Time-series forecasting using statistical, neural, and ensemble architectures. Frontiers Big Data, 3, 2020
2020
-
[16]
Kingma and Jimmy Ba
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Interna- tional Conference on Learning Representations, 2015
2015
-
[17]
Towards long-term time-series forecasting: Feature, pattern, and distribution
Yan Li, Xinjiang Lu, Haoyi Xiong, Jian Tang, Jiantao Su, Bo Jin, and Dejing Dou. Towards long-term time-series forecasting: Feature, pattern, and distribution. In IEEE International Conference on Data Engineering, 2023. 10
2023
-
[18]
Time series forecasting with deep learning: A survey
Bryan Lim and Stefan Zohren. Time series forecasting with deep learning: A survey. arXiv, 2004.13408, 2020
2004 arXiv
-
[19]
itransformer: Inverted transformers are effective for time series forecasting
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. In International Conference on Learning Representations, 2024
2024
-
[20]
Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam
Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations, 2023
2023
-
[21]
Vale, and José Silva
Tiago Pinto, Isabel Praça, Zita A. Vale, and José Silva. Ensemble learning for electricity consumption forecasting in office buildings. Neurocomputing, 423, 2021
2021
-
[22]
Latent ordinary differential equations for irregularly-sampled time series
Yulia Rubanova, Tian Qi Chen, and David Duvenaud. Latent ordinary differential equations for irregularly-sampled time series. In Annual Conference on Neural Information Processing Systems, 2019
2019
-
[23]
Modeling irregular time series with continuous recurrent units
Mona Schirmer, Mazin Eltayeb, Stefan Lessmann, and Maja Rudolph. Modeling irregular time series with continuous recurrent units. In International Conference on Machine Learning, 2022
2022
-
[24]
Satya Narayan Shukla and Benjamin M. Marlin. A survey on principles, models and methods for learning from irregularly sampled time series: From discretization to attention and invariance. arXiv, 2012.00168, 2020
2012 arXiv
-
[26]
An analysis of linear time series forecasting models
William Toner and Luke Nicholas Darlow. An analysis of linear time series forecasting models. In International Conference on Machine Learning, 2024
2024
-
[27]
Gomez, Lukasz Kaiser, and Illia Polosukhin
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Annual Conference on Neural Information Processing Systems, 2017
2017
-
[28]
Rethinking the power of timestamps for robust time series forecasting: A global-local fusion perspective
Chengsen Wang, Qi Qi, Jingyu Wang, Haifeng Sun, Zirui Zhuang, Jinming Wu, and Jianxin Liao. Rethinking the power of timestamps for robust time series forecasting: A global-local fusion perspective. In Annual Conference on Neural Information Processing Systems, 2024
2024
-
[29]
Timesnet: Temporal 2d-variation modeling for general time series analysis
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In International Conference on Learning Representations, 2023
2023
-
[30]
Not only pairwise relationships: Fine-grained relational modeling for multivariate time series forecasting
Jinming Wu, Qi Qi, Jingyu Wang, Haifeng Sun, Zhikang Wu, Zirui Zhuang, and Jianxin Liao. Not only pairwise relationships: Fine-grained relational modeling for multivariate time series forecasting. In International Joint Conference on Artificial Intelligence, 2023
2023
-
[31]
Graph wavenet for deep spatial-temporal graph modeling
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial-temporal graph modeling. In International Joint Conference on Artificial Intelligence, 2019
2019
-
[32]
Connecting the dots: Multivariate time series forecasting with graph neural networks
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Connecting the dots: Multivariate time series forecasting with graph neural networks. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2020
2020
-
[33]
Grafiti: Graphs for forecasting irregularly sampled time series
Vijaya Krishna Yalavarthi, Kiran Madhusudhanan, Randolf Scholz, Nourhan Ahmed, Johannes Burchert, Shayan Jawed, Stefan Born, and Lars Schmidt-Thieme. Grafiti: Graphs for forecasting irregularly sampled time series. In AAAI Conference on Artificial Intelligence, 2024
2024
-
[34]
Fouriergnn: Rethinking multivariate time series forecasting from a pure graph perspective
Kun Yi, Qi Zhang, Wei Fan, Hui He, Liang Hu, Pengyang Wang, Ning An, Longbing Cao, and Zhendong Niu. Fouriergnn: Rethinking multivariate time series forecasting from a pure graph perspective. In Annual Conference on Neural Information Processing Systems, 2023
2023
-
[35]
Is channel independent strategy optimal for time series forecasting? arXiv, 2310.17658, 2023
Peiwen Yuan and Changsheng Zhu. Is channel independent strategy optimal for time series forecasting? arXiv, 2310.17658, 2023. 11
2023 arXiv
-
[36]
Are transformers effective for time series forecasting? In AAAI Conference on Artificial Intelligence, 2023
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In AAAI Conference on Artificial Intelligence, 2023
2023
-
[37]
Warpformer: A multi-scale modeling approach for irregular clinical time series
Jiawen Zhang, Shun Zheng, Wei Cao, Jiang Bian, and Jia Li. Warpformer: A multi-scale modeling approach for irregular clinical time series. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2023
2023
-
[38]
Unleash the power of pre-trained language models for irregularly sampled time series
Weijia Zhang, Chenlong Yin, Hao Liu, and Hui Xiong. Unleash the power of pre-trained language models for irregularly sampled time series. arXiv, 2408.08328, 2024
2024 arXiv
-
[39]
Irregular multivari- ate time series forecasting: A transformable patching graph neural networks approach
Weijia Zhang, Chenlong Yin, Hao Liu, Xiaofang Zhou, and Hui Xiong. Irregular multivari- ate time series forecasting: A transformable patching graph neural networks approach. In International Conference on Machine Learning, 2024
2024
-
[40]
Irregular traffic time series forecasting based on asynchronous spatio-temporal graph convolutional networks
Weijia Zhang, Le Zhang, Jindong Han, Hao Liu, Yanjie Fu, Jingbo Zhou, Yu Mei, and Hui Xiong. Irregular traffic time series forecasting based on asynchronous spatio-temporal graph convolutional networks. In ACM SIGKDD International Conference on Knowledge Discovery and Data Min...
2024
-
[41]
Graph-guided network for irregularly sampled multivariate time series
Xiang Zhang, Marko Zeman, Theodoros Tsiligkaridis, and Marinka Zitnik. Graph-guided network for irregularly sampled multivariate time series. In International Conference on Learning Representations, 2022
2022
-
[42]
Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting
Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In International Conference on Learning Representa- tions, 2023
2023
-
[43]
Rethinking channel dependence for multivariate time series forecasting: Learning from leading indicators
Lifan Zhao and Yanyan Shen. Rethinking channel dependence for multivariate time series forecasting: Learning from leading indicators. In International Conference on Learning Representations, 2024
2024
-
[44]
Forecasting fine-grained air quality based on big data
Yu Zheng, Xiuwen Yi, Ming Li, Ruiyuan Li, Zhangqing Shan, Eric Chang, and Tianrui Li. Forecasting fine-grained air quality based on big data. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015. 12 A Detailed Analysis of ALinear Algorithm 1 ALin...
2015
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.