Machine Learning · arXiv:2611.16514 v1 · submitted 12 Nov 2026 · 54 references
Learning Temporal Dynamics with Derivative Representations for Time Series Forecasting
The paper says
“Time series forecasting aims to predict the future evolution of complex systems by learning their underlying dynamics from past observations. Despite recent advances in deep learning, Transformers in particular still struggle to capture the temporal evolution of system dynamics.”
more of the abstract
Existing approaches have been largely driven by architectural innovation, with comparatively little attention paid to training strategies that explicitly exploit the temporal structure governing system dynamics. In this work, we introduce DeltaTSF, a dynamics-aware forecasting framework based on discrete derivative representations of time series. Specifically, DeltaTSF jointly learns the series and their discrete temporal derivatives across both input and forecast horizons within a unified, model-agnostic framework. By coupling these representations over time, the model is encouraged to maintain temporal consistency, a property we show to induce a form of discrete Taylor consistency during training. This formulation enables the model to better reflect the underlying system dynamics, resulting in more stable and physically coherent forecasts. Notably, DeltaTSF does not require explicit derivative computation and introduces no additional computational overhead at inference time. Extensive experiments across diverse benchmarks and backbone architectures demonstrate that DeltaTSF consistently improves forecasting performance and yields more dynamically coherent predictions than strong SOTA methods. These results highlight the effectiveness of enforcing dynamic coherence through derivative representations as a model-agnostic paradigm that operates alongside and complements architectural design. Source code will be made publicly available upon acceptance.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 13 Nov 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. DeltaTSF is a well-motivated training trick that consistently improves forecasting accuracy, but the fixed 720-step lookback in experiments needs a robustness check before the claimed advantage is solid. This line is wrong
4 major objections · 6 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 54 references say.
The paper says, at §4.1, Tables 2–3
The review did not quote this passage.
Pith says: Objection 1 of 4 · major
The input sequence length is fixed to 720 for all methods, including baselines originally validated with shorter lookback windows (e.g., 336 or 96). Without retuning hyperparameters for the longer lookback, baselines may underperform, while DeltaTSF could benefit from longer context (auxiliary losses use the full window).
more of objection 1
The paper does not provide main-table results for alternative input lengths (e.g., 336) to verify robustness. Table 6 shows partial results at length 336 for three datasets against PSLoss, but this does not cover the full baseline set. To support the central claim of consistent improvement, the authors should either (a) report main results with input length 336 for all baselines that were originally tuned for that setting, or (b) demonstrate that performance is insensitive to input length by providing a sensitivity analysis.
The paper says, at §2, §4.2, Tables 1 and 3 (PhaseFormer baseline)
The review did not quote this passage.
Pith says: Objection 2 of 4 · major
The paper lists 'PhaseFormer' as a time series forecasting baseline and cites Ref. [26] (Khan et al., 'Phaseformer: Phase-based attention mechanism for underwater image restoration and beyond', WACV 2025). The cited publication addresses underwater image restoration, not time series forecasting.
more of objection 2
If the planning-to-forecasting adaptation is not described in the paper, the baseline results are unsupported. The authors must clarify how PhaseFormer was applied to time series and whether the reported numbers are from a valid time series implementation, or correct the baseline selection.
The paper says, at Appendix A.1
The review did not quote this passage.
Pith says: Objection 3 of 4 · major
The derivation claims that the training framework 'guarantees' Taylor consistency (line after Eq. 18). However, the preceding derivation explicitly ignores higher-order terms in the Taylor expansion and treats the finite differences as exact derivatives. In practice, both the truncation and discretization errors are uncontrolled, especially for noisy or non-smooth time series.
more of objection 3
The wording should be softened to 'encourages' or 'approximates' to reflect the approximation inherent in finite-difference training. This is not a fatal flaw but a precision issue that affects how the method's theoretical grounding is presented.
The paper says, at §3.5, Table 4b
The review did not quote this passage.
Pith says: Objection 4 of 4 · major
The Compute-Equivalent Sampling (CES) strategy is evaluated only with a subset of datasets and backbones. The paper asserts a computational savings of 'at least 50%', but does not report actual training time or FLOPs. Without a concrete runtime comparison, the practical benefit of CES is unclear. Recommend including wall-clock time per epoch or relative FLOPs for the full setting (with and without CES) for a representative backbone and dataset.
6 minor objections
The paper says, at Equation (12)
Pith says: minor objection 1 of 6
The loss function writes λ_k multiplied by the Frobenius norm, but the note at Eq. 2 uses ∥·∥_2^2. For concistency, define whether the norm is over the entire matrix or each example. As written, it implies a batch average outside but the norm is over the horizon and variables. Consider rewriting as (1/B) Σ_b ∥_ŷ - y∥_F^2 to clarify.
The paper says, at Figure 1
Pith says: minor objection 2 of 6
The figure illustrates input, backbone, and output but does not explicitly show the derivative computation step or the multiple loss branches. Adding a label for each derivative order loss and the shared backbone would improve readability.
show 4 more
The paper says, at Table 1
Pith says: minor objection 3 of 6
The taxonomy column 'Derivative (Rate of Change)' lists only our method. However, state-space models and Neural ODEs also model derivatives implicitly. Consider adding a note or row for dynamical methods even if they are not baselines, to provide context.
The paper says, at §C.2
Pith says: minor objection 4 of 6
The learning rate is fixed at 1×10^{-4] and batch size to 32 (or 128). Are these values tuned separately for each dataset/backbone? If not, state that they are fixed to avoid over-tuning concerns.
The paper says, at Table 4a
Pith says: minor objection 5 of 6
Label the columns more clearly: 'OS+LS' and 'Only IS' and 'Only LS'. Currently the abbreviations may confuse readers unfamiliar with the notation. Define them in the caption.
The paper says, at Ref. [26]
Pith says: minor objection 6 of 6
As noted in major comments, the PhaseFormer paper is about underwater image restoration. If the baseline is a different work or the application was repurposed, please update the reference and description accordingly.
What would settle it
Re-run Table 3 (average MSE/MAE over four horizons) for all methods with input length set to 336 (the most common setting used in baseline papers). Compute the average relative improvement of DeltaTSF over the best baseline for each dataset. If the improvement falls below 5% or becomes statistically non-significant (via paired bootstrap, n=200), the claim of consistent superiority across benchmarks is undermined.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 13 Nov 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 4 major objections, 6 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Here is my take on arXiv:2611.16514.
The paper proposes a simple training-time augmentation: add auxiliary losses on the first- and possibly second-order finite differences of the target sequence, computed from both input and forecast windows, while sharing the same backbone. The idea is clean, model-agnostic, and the loss can be dropped at inference. The authors show that this consistently reduces MSE and MAE across several recent backbones (PatchTST, iTransformer, TimesNet, TimeMixer) on six standard benchmarks. The ablations are careful—they decompose where to inject derivative information (input, output, loss) and confirm that all three together help most. The compute-equivalent sampling trick keeps training cost comparable to baseline. This is a solid empirical contribution.
The main concern that gives me pause is the experimental setup. The paper fixes the input lookback to 720 steps for every method. Many baselines (PatchTST, iTransformer, DLinear) were validated on shorter windows (96 or 336) and their official hyperparameters are tuned for those lengths. Using 720 without retuning may systematically hurt the baselines while DeltaTSF may benefit from longer context because its derivative computation eats into the effective length (mitigated by zero-padding). The paper does not report results for alternative lookback lengths for the main tables, so we cannot rule out that part of the reported gain is an artifact of this asymmetric design. The authors do compare with PSLoss at input length 336 in Table 6, but this covers only one setting and one competitor.
Other soft spots: no error bars (single runs only, no standard deviation) and no code at submission. These are standard now for empirical ML papers and should be addressed.
The theoretical claim of “Taylor consistency” is approximate (truncated series) and not rigorously shown to hold beyond the loss definition—but the paper frames it as an interpretation, not a proof, so this is acceptable.
On balance: the core idea is sensible, the experiments are otherwise thorough, and the results are consistent. The input-length concern is real enough that I would want to see it addressed before relying on the claimed advantage, but it does not invalidate the approach. I would accept a peer-review invitation for this paper; the reviewers should ask for robustness checks on lookback length and for error bars/code.
Best, [Your name]
The strongest objection
The experimental comparisons in Tables 2 and 3 fix the input sequence length to 720 for every method (stated in §4.1: 'the input sequence length is fixed to 720 for all methods'). Many widely used baselines (e.g., PatchTST, iTransformer, DLinear) were originally validated with input lengths of 336 or 96, and their official hyperparameters were tuned for those settings. Using a substantially longer lookback without retuning may degrade their performance, while DeltaTSF could benefit from longer inputs because derivative computation reduces effective length (padding compensates) and the auxiliary losses may exploit longer context. The paper does not report results for alternative input lengths (e.g., 336) to verify that the observed improvements are robust. Table 6 shows a comparison with PSLoss at input length 336 where DeltaTSF is competitive, but this does not cover the main tables. Without controlling for this design choice, the central claim of 'consistently improves forecasting performance' may be an artifact of a favorable experimental configuration for DeltaTSF, rather than a genuine advantage of derivative-based training.
The reader identified a valid theoretical concern about discrete differences capturing dynamics, but the most load-bearing threat to the central claim is an experimental design choice—fixing input length to 720—that may systematically favor DeltaTSF over baselines. The paper presents no evidence that baselines were re-tuned for this length; the statement 'careful tuning to ensure fair comparison' is vague. While the reader called for error bars and code, the input-length confound is more immediate: if the relative gains evaporate at shorter lookbacks, the method's practical value is narrower than claimed. The verdict remains CONDITIONAL, but the condition should explicitly require verification at multiple input lengths. My partial agreement stems from the fact that the reader's weakest assumption (sufficiency of discrete derivatives) is distinct and more theoretical, whereas my concern is empirical and directly testable with existing code. I do not recommend changing the verdict because the paper has substantial empirical support (consistent gains across four backbones, multiple datasets) and the concern can be resolved without new experiments—re-running at input length 336 is straightforward. If the authors provide those results and they align with the current conclusions, the claim stands strengthened.
The referee's report
If the reported gains are robust and general, the work offers a simple, drop-in training loss that improves forecasting accuracy without modifying model architectures. This is appealing for practitioners and complements the current emphasis on architectural innovation. The idea of enforcing temporal coherence via derivative representations is intuitive and has plausible connections to continuous-time dynamics. However, the strength of the evidence is weakened by a confound in the experimental design (uniform input length 720 favoring longer lookback) and a likely misattributed baseline (PhaseFormer, originally for image restoration). The magnitude of improvement is modest (e.g., ~5% relative MSE reduction on ETTh1) but consistent, which could still be practically meaningful. The method's model-agnostic nature and zero inference overhead are clear strengths.
Pith's reading of the paper (a machine's summary, not the authors')
The paper aims to show that enforcing temporal consistency through derivative representations is an effective and general training strategy for time series forecasting, complementing architectural innovation. DeltaTSF jointly learns the series and its discrete first- and second-order finite differences across both input and forecast horizons within a single, unmodified backbone. The coupling of these derivative orders induces an approximate commutativity between the forecaster and temporal differencing operators, which the authors interpret as discrete Taylor consistency. This regularization stabilizes long-horizon predictions and reduces error accumulation. Across six benchmarks and four backbone architectures, DeltaTSF consistently improves MSE and MAE over strong baselines, with negligible added training cost and zero inference overhead.
What carries the argument. The key machinery is a multi-order derivative loss L(θ) = Σ_{k=0}^K λ_k E[||f_θ($x^{{(k)}}$) - $y^{{(k)}}$||^2_2], where $x^{{(k)}}$ and $y^{{(k)}}$ are the k-th order finite-difference representations of the input and target sequences, computed via linear difference matrices D_k. The shared backbone f_θ processes all derivative orders identically, while a Compute-Equivalent Sampling (CES) strategy stochastically selects a single derivative order per sample during training, keeping computational cost equivalent to the base model. This design induces gradient coupling across temporal orders and enforces an approximate commutativity between the forecaster and differencing operators.
Core claim. The core claim is that augmenting standard value-only supervision with auxiliary losses on discrete temporal derivatives (up to second order) forces the predictor to respect the local dynamical structure of the underlying continuous process. The paper shows that this training framework implicitly enforces an approximate commutativity: Δ^k f_θ(x) ≈ f_θ(Δ^k x) for k = 1,…,K, which is exactly a discrete truncated Taylor expansion across the forecast horizon. The authors demonstrate empirically that this derivative coupling, applied jointly at input, output, and loss stages, consistently improves forecasting accuracy and yields predictions with more coherent temporal dynamics than existing methods, including those with specialized architectures.
The premise it rests on. The paper assumes that discrete finite differences (especially first-order) are a sufficient proxy for the underlying continuous temporal dynamics, and that aligning these differences across input and output via a shared model leads to better generalization, without theoretically justifying that this assumption holds for arbitrary time series.
If the paper is right (Pith's inferences, not the authors')
Any existing time series forecasting backbone — Transformer, MLP, CNN, or linear — can be improved by adding DeltaTSF's derivative supervision without architectural changes, making it a drop-in enhancement.
The derivative coupling reduces error accumulation in multi-step forecasts, leading to more stable long-horizon predictions, especially for horizons up to 720 steps.
The framework is robust to higher-order differences (K=2,3,4), which may add noise but do not degrade performance, suggesting that practitioners can default to K=1 with confidence.
Because DeltaTSF requires no derivative computation at inference, it can be deployed in latency-sensitive applications where only the raw prediction is needed.
The method provides a baseline for future work that explicitly designs architectures to exploit derivative representations, as the authors suggest in the limitations.
DeltaTSF may be particularly effective for systems with smooth underlying dynamics (e.g., physical measurements, energy load, weather) where low-order derivatives carry meaningful signal; it may offer less benefit on highly stochastic or noisy series, a boundary condition the paper does not explore.
The derivative coupling could be extended beyond finite differences to more robust derivative estimates (e.g., Savitzky–Golay filters) to reduce noise amplification, especially at higher orders.
The principle of imposing temporal consistency through derivative supervision may transfer to other sequence tasks such as anomaly detection, imputation, or causal discovery, where dynamics matter beyond pointwise accuracy.
The CES strategy, while effective, introduces randomness that may interact differently with different backbones; a systematic study of CES probability tuning could further optimize the accuracy–cost trade-off.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Time series forecasting aims to predict the future evolution of complex systems by learning their underlying dynamics from past observations. Despite recent advances in deep learning, Transformers in particular still struggle to capture the temporal evolution of system dynamics. Existing approaches have been largely driven by architectural innovation, with comparatively little attention paid to training strategies that explicitly exploit the temporal structure governing system dynamics. In this work, we introduce DeltaTSF, a dynamics-aware forecasting framework based on discrete derivative representations of time series. Specifically, DeltaTSF jointly learns the series and their discrete temporal derivatives across both input and forecast horizons within a unified, model-agnostic framework. By coupling these representations over time, the model is encouraged to maintain temporal consistency, a property we show to induce a form of discrete Taylor consistency during training. This formulation enables the model to better reflect the underlying system dynamics, resulting in more stable and physically coherent forecasts. Notably, DeltaTSF does not require explicit derivative computation and introduces no additional computational overhead at inference time. Extensive experiments across diverse benchmarks and backbone architectures demonstrate that DeltaTSF consistently improves forecasting performance and yields more dynamically coherent predictions than strong SOTA methods. These results highlight the effectiveness of enforcing dynamic coherence through derivative representations as a model-agnostic paradigm that operates alongside and complements architectural design. Source code will be made publicly available upon acceptance.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-13
The authors' main claim, as the classifier picked it out 0.66
“These results highlight the effectiveness of enforcing dynamic coherence through derivative representations as a model-agnostic paradigm that operates alongside and complements architectural design.”
How the main claim is stated: asserted 0.96
What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.15 · human participants 0.03 · controlled comparison 0.66 · randomized assignment 0.01 · states quantitative result 0.05 · compares with prior results 0.94 · mentions code or data availability 0.65
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Time series forecasting aims to predict the future evolution of complex systems by learning their underlying dynamics from past observations. | background 1.00 | 0.05 | 0.02 | asserted 0.99 |
| Despite recent advances in deep learning, Transformers in particular still struggle to capture the temporal evolution of system dynamics. | gap 0.86 | 0.07 | 0.02 | asserted 0.99 |
| Existing approaches have been largely driven by architectural innovation, with comparatively little attention paid to training strategies that explicitly exploit the temporal structure governing system dynamics. | gap 0.98 | 0.07 | 0.03 | asserted 1.00 |
| In this work, we introduce DeltaTSF, a dynamics-aware forecasting framework based on discrete derivative representations of time series. | contribution 0.97 | 0.90 | 0.08 | asserted 1.00 |
| Specifically, DeltaTSF jointly learns the series and their discrete temporal derivatives across both input and forecast horizons within a unified, model-agnostic framework. | method 0.94 | 0.91 | 0.07 | asserted 1.00 |
| By coupling these representations over time, the model is encouraged to maintain temporal consistency, a property we show to induce a form of discrete Taylor consistency during training. | method 0.75 | 0.90 | 0.09 | asserted 0.97 |
| This formulation enables the model to better reflect the underlying system dynamics, resulting in more stable and physically coherent forecasts. | interpretation 0.78 | 0.69 | 0.21 | asserted 0.96 |
| Notably, DeltaTSF does not require explicit derivative computation and introduces no additional computational overhead at inference time. | method 0.48 | 0.86 | 0.05 | asserted 1.00 |
| Extensive experiments across diverse benchmarks and backbone architectures demonstrate that DeltaTSF consistently improves forecasting performance and yields more dynamically coherent predictions than strong SOTA methods. | result 1.00 | 0.96 | 0.68 | asserted 1.00 |
| These results highlight the effectiveness of enforcing dynamic coherence through derivative representations as a model-agnostic paradigm that operates alongside and complements architectural design. | interpretation 0.99 | 0.78 | 0.73 | asserted 0.96 |
| Source code will be made publicly available upon acceptance. | future work 0.95 | 0.16 | 0.02 | asserted 0.95 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.89
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-13: 1 request, 8,087 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (3)
Figures are the authors', reproduced from the paper.
References (54)
- [26] Raqib Khan, Anshul Negi, Ashutosh Kulkarni, Shruti S Phutke, Santosh Kumar Vipparthi, and Subrah- manyam Murala. Phaseformer: Phase-based attention mechanism for underwater image restoration and beyond. InProceedings of the Winter Conference on Applications of Computer Vision,... on Pith
- [1] Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Oriol Vinyals, Jacklynn Stott, Alexander Pritzel, Shakir Mohamed, and P... on Pith
- [2] Haixu Wu, Hang Zhou, Mingsheng Long, and Jianmin Wang. Interpretable weather forecasting for worldwide stations with a unified deep model.Nature Machine Intelligence, 5(6):602–611, 2023 on Pith
- [3] Razvan-Gabriel Cirstea, Bin Yang, Chenjuan Guo, Tung Kieu, and Shirui Pan. Towards spatio-temporal aware traffic time series forecasting. In2022 IEEE 38th International Conference on Data Engineering (ICDE), pages 2900–2913. IEEE, 2022 on Pith
- [4] Artur Trindade. ElectricityLoadDiagrams20112014. UCI Machine Learning Repository, 2015. DOI: https://doi.org/10.24432/C58C86 on Pithdoi:10.24432/c58c86
- [5] Hongbin Huang, Minghua Chen, and Xiao Qiao. Generative learning for financial time series with irregular and scale-invariant patterns. InThe Twelfth International Conference on Learning Representations, 2024 on Pith
- [6] Yanru Sun, Zongxia Xie, Emadeldeen Eldele, Dongyue Chen, Qinghua Hu, and Min Wu. Learning pattern-specific experts for time series forecasting under patch-level distribution shift.arXiv preprint arXiv:2410.09836, 2024 on PitharXiv
- [7] Chengqing Yu, Fei Wang, Zezhi Shao, Tangwen Qian, Zhao Zhang, Wei Wei, and Yongjun Xu. Ginar: An end-to-end multivariate time series forecasting model suitable for variable missing. InProceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pages 3... on Pith
- [8] Xiangfei Qiu, Xingjian Wu, Yan Lin, Chenjuan Guo, Jilin Hu, and Bin Yang. Duet: Dual clustering enhanced multivariate time series forecasting. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V . 1, pages 1185–1196, 2025 on Pith
- [9] Yanru Sun, Zongxia Xie, Haoyu Xing, Hualong Yu, and Qinghua Hu. Ppgf: Probability pattern-guided time series forecasting.IEEE Transactions on Neural Networks and Learning Systems, 2025 on Pith
- [10] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? InProceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121–11128, 2023 on Pith
- [11] Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers, 2023 on Pith
- [12] Dilfira Kudrat, Zongxia Xie, Yanru Sun, Tianyu Jia, and Qinghua Hu. Patch-wise structural loss for time series forecasting. InForty-second International Conference on Machine Learning, 2025 on Pith
- [13] Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Olek- sandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, et al. Chronos: Learning the language of time series.arXiv preprint arXiv:2403.07815, 2024. 10 on PithPith's reviewarXiv
- [14] Siru Zhong, Weilin Ruan, Ming Jin, Huan Li, Qingsong Wen, and Yuxuan Liang. Time-vlm: Exploring mul- timodal vision-language models for augmented time series forecasting.arXiv preprint arXiv:2502.04395, 2025 on PithPith's reviewarXiv
- [15] Yekun Ke, Yingyu Liang, Zhenmei Shi, Zhao Song, and Chiwun Yang. Curse of attention: A kernel-based perspective for why transformers fail to generalize on time series forecasting and beyond.arXiv preprint arXiv:2412.06061, 2024 on PithPith's reviewarXiv
- [16] Zida Liang, Jiayi Zhu, and Weiqiang Sun. Why attention fails: The degeneration of transformers into mlps in time series forecasting.arXiv preprint arXiv:2509.20942, 2025 on PitharXiv
- [17] Vijay Ekambaram, Arindam Jati, Nam Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. Tsmixer: Lightweight mlp-mixer model for multivariate time series forecasting. InProceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining, pages 459–469, 2023 on Pith
- [18] Jongseon Kim, Hyungjoon Kim, HyunGi Kim, Dongjun Lee, and Sungroh Yoon. A comprehensive survey of deep learning for time series forecasting: architectural diversity and open challenges.Artificial Intelligence Review, 58(7):1–95, 2025 on Pith
- [19] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itrans- former: Inverted transformers are effective for time series forecasting.arXiv preprint arXiv:2310.06625, 2023 on PithPith's reviewarXiv
show 34 more
- [20] Qihe Huang, Zhengyang Zhou, Kuo Yang, Zhongchao Yi, Xu Wang, Wu Jiang, and Yang Wang. Timebase: The power of minimalism in long-term time series forecasting. InForty-second International Conference on Machine Learning, 2025 on Pith
- [21] Yuxin Chang, Alex James Boyd, Cao Xiao, Taha Kass-Hout, Parminder Bhatia, Padhraic Smyth, and Andrew Warrington. Deep continuous-time state-space models for marked event sequences. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025 on Pith
- [22] Jiaxi Hu, Disen Lan, Ziyu Zhou, Qingsong Wen, and Yuxuan Liang. Time-ssm: Simplifying and unifying state space models for time series forecasting.arXiv preprint arXiv:2405.16312, 2024 on PithPith's reviewarXiv
- [23] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis.arXiv preprint arXiv:2210.02186, 2022 on PithPith's reviewarXiv
- [24] Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and Jun Zhou. Timemixer: Decomposable multiscale mixing for time series forecasting.arXiv preprint arXiv:2405.14616, 2024 on PithPith's reviewarXiv
- [25] Peiwang Tang and Weitai Zhang. Unlocking the power of patch: Patch-based mlp for long-term time series forecasting. InProceedings of the AAAI conference on artificial intelligence, volume 39, pages 12640–12648, 2025 on Pith
- [27] Wentian Zhao, Yanyun Gao, Tingxiang Ji, Xili Wan, Feng Ye, and Guangwei Bai. Deep temporal convolutional networks for short-term traffic flow forecasting.Ieee Access, 7:114496–114507, 2019 on Pith
- [28] Shumin Deng, Ningyu Zhang, Wen Zhang, Jiaoyan Chen, Jeff Z Pan, and Huajun Chen. Knowledge-driven stock trend prediction and explanation via temporal convolutional network. InCompanion proceedings of the 2019 world wide web conference, pages 678–685, 2019 on Pith
- [29] David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic forecasting with autoregressive recurrent networks.International journal of forecasting, 36(3):1181–1191, 2020 on Pith
- [30] Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. xlstm: Extended long short-term memory.Advances in Neural Information Processing Systems, 37:107547–107603, 2024 on Pith
- [31] Ioannis E Livieris, Emmanuel Pintelas, and Panagiotis Pintelas. A cnn–lstm model for gold price time-series forecasting.Neural computing and applications, 32(23):17351–17360, 2020 on Pith
- [32] Harya Widiputra, Adele Mailangkay, and Elliana Gautama. Multivariate cnn-lstm model for multiple parallel financial time-series prediction.Complexity, 2021(1):9903518, 2021. 11 on Pith
- [33] Albert Gu, Karan Goel, and Christopher Re. Efficiently modeling long sequences with structured state spaces. InInternational Conference on Learning Representations, 2021 on Pith
- [34] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations.Advances in neural information processing systems, 31, 2018 on Pith
- [35] C Rackauckas, Y Ma, J Martensen, C Warner, K Zubov, R Supekar, D Skinner, A Ramadhan, and A Edelman. Universal differential equations for scientific machine learning. 2020 on Pith
- [36] Junteng Jia and Austin R Benson. Neural jump stochastic differential equations.Advances in Neural Information Processing Systems, 32, 2019 on Pith
- [37] Leena Chennuru Vankadara, Jin Xu, Moritz Haas, and V olkan Cevher. On feature learning in structured state space models. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024 on Pith
- [38] Patrick Kidger.On Neural Differential Equations. PhD thesis, University of Oxford, 2021 on Pith
- [39] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. InFirst conference on language modeling, 2024 on Pith
- [40] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017 on Pith
- [41] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. InProceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106–11115, 2021 on Pith
- [42] Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. InThe eleventh international conference on learning representations, 2023 on Pith
- [43] Zhenwei Zhang, Linghang Meng, and Yuantao Gu. Sageformer: Series-aware framework for long-term multivariate time-series forecasting.IEEE Internet of Things Journal, 11(10):18435–18448, 2024 on Pith
- [44] Kashif Rasul, Arjun Ashok, Andrew Robert Williams, Arian Khorasani, George Adamopoulos, Rishika Bhagwatkar, Marin Biloš, Hena Ghonia, Nadhir Hassen, Anderson Schneider, et al. Lag-llama: Towards foundation models for time series forecasting. InR0-FoMo: Robustness of Few-shot a... on Pith
- [45] Shashank Yadav and Vignesh Subbian. When attention fails: Pitfalls of attention-based model interpretabil- ity for high-dimensional clinical time-series. InConference on Health, Inference, and Learning, pages 289–305. PMLR, 2025 on Pith
- [46] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020 on Pith
- [47] Marcel Kollovieh, Abdul Fatir Ansari, Michael Bohlke-Schneider, Jasper Zschiegner, Hao Wang, and Yuyang Bernie Wang. Predict, refine, synthesize: Self-guiding diffusion models for probabilistic time series forecasting.Advances in Neural Information Processing Systems, 36:28341... on Pith
- [48] Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score-based diffusion models for probabilistic time series imputation.Advances in neural information processing systems, 34:24804–24816, 2021 on Pith
- [49] Juan Miguel Lopez Alcaraz and Nils Strodthoff. Diffusion-based time series imputation and forecasting with structured state space models.arXiv preprint arXiv:2208.09399, 2022 on PithPith's reviewarXiv
- [50] Xiangfei Qiu, Xingjian Wu, Hanyin Cheng, Xvyuan Liu, Chenjuan Guo, Jilin Hu, and Bin Yang. Dbloss: Decomposition-based loss function for time series forecasting. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025 on Pith
- [51] Junwoo Park, Daehoon Gwak, Jaegul Choo, and Edward Choi. Self-supervised contrastive forecasting. CoRR, 2024 on Pith
- [52] Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances in neural information processing systems, 32, 2019. 12 on Pith
- [53] Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. InInternational conference on learning representations, 2021 on Pith
- [54] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. InInternational Conference on Learning Representations, 2018. 13 A Analysis of DeltaTSF A.1 Taylor Consistency Induced by Our Training Framework Let us consider a loc... on Pith
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment