REVIEW 4 major objections 5 minor 28 references
Time Series Foundational Models: Their Role in Anomaly Detection and Prediction
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper claims that time-series foundation models, despite their forecasting strengths, generally do not beat traditional statistical and deep-learning baselines on anomaly detection and prediction, and are much more expensive.
desk verdict A useful but sloppy benchmark: the central caution about TSFMs is plausible, but the anomaly-prediction numbers compare different tasks and the tables have contradictions. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The evaluation design is the paper's central mechanism. It selects TSFMs only if they are explicitly designed and trained for anomaly detection or forecasting on multivariate data, then applies each model under its original hyperparameters, replicates zero-shot and fine-tuned settings, and maps forecast outputs to anomaly labels through fixed feature thresholds before comparing all models on identical train/test splits. The baselines—weighted XGBoost and an autoencoder—carry the argument because they are trained from scratch on the same data and provide the accuracy-efficiency point of comparison.
What would settle it
A re-run of the same five datasets in which TimeGPT, MOIRAI, or Chronos is given a purpose-built anomaly head or an anomaly-aware loss, with thresholds chosen only from training data, and then beats weighted XGBoost's Pulp F1 of 0.94 or its SMD F1 of 0.87 at comparable inference time, would directly contradict the paper's blanket conclusion.
Extended reading notes
Core claim
The paper's core claim is that TSFMs, while competitive in pure forecasting, are not specialized for anomaly work and lose to simpler models when adapted to it. Only TimeGPT and FPT have built-in anomaly-detection behavior among the five reference TSFMs, and none is designed for anomaly prediction, so the authors adapt next-step forecasting outputs into anomaly predictions by comparing forecasts against predefined safe thresholds or by setting the anomaly label as the forecasting target. Under that adaptation, weighted XGBoost and an autoencoder achieve equal or better precision, recall, F1, and forecasting error on the Pulp, Future Factories, SMD, MSL, and ETTh1 datasets, with inference times under a minute to a few minutes versus tens of minutes for several TSFMs. The TSFMs also fail to show consistent zero-shot or few-shot advantages and require high computational resources.
Load-bearing premise
The conclusion depends on treating thresholded forecasts—or forecasting the anomaly label—as a faithful use of each TSFM's anomaly-prediction ability, equivalent to a supervised classifier.
Editorial extensions
If this is right
- Anomaly-detection practitioners should benchmark simple statistical and deep-learning baselines before adopting a TSFM, because the paper finds the baselines match or exceed TSFM accuracy on five datasets.
- Fine-tuning a TSFM is not an automatic win for anomaly work: the paper reports only marginal zero-shot-to-finetuned improvements across TimeGPT, MOIRAI, and Chronos.
- The absence of a TSFM purpose-built for anomaly prediction means that forecast-to-threshold conversion is the standard route, and that route is less accurate and slower than the baselines in this study.
- Computational budget matters: inference times for FPT and TimeGPT reach tens of minutes, while the baselines finish in under a minute on the same tasks, changing the practical cost equation even where accuracy is close.
Reading between the lines
- Beyond the paper, the negative result may reflect a task mismatch rather than a fundamental ceiling on TSFMs; an anomaly-aware pretraining objective or a classification head on top of TSFM embeddings could close or reverse the gap.
- Beyond the paper, the forecast-to-threshold procedure needs externally defined feature thresholds, which are rarely available in new deployments; a learned-threshold version is a concrete next experiment that would test whether the TSFM disadvantage persists.
- Beyond the paper, the compute comparison is in inference minutes only; a full lifecycle cost including pretraining energy and fine-tuning would likely tilt the balance even further toward the baselines.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper evaluates five time series foundation models (TimeGPT, FPT, Time-MOE, MOIRAI, Chronos) for anomaly detection and anomaly prediction on five datasets (Future Factories, Pulp, SMD, MSL, ETTh1), comparing zero-shot and fine-tuned performance against a weighted XGBoost baseline and an autoencoder. The authors report that TSFMs can be adapted for anomaly tasks, but that traditional statistical and deep learning models often match or outperform them, and that TSFMs incur higher computational costs. The paper contributes benchmark tables, a computational-cost analysis, and a GitHub repository with preprocessed data and code.
Significance. If the central claim held, the paper would be a valuable cautionary benchmark: practitioners should not assume pretrained TSFMs are superior for anomaly detection and prediction, and should compare against simple baselines. The study is broad (five models, five datasets, zero-shot and fine-tuning, computational cost) and the authors provide reproducibility artifacts. However, the validity of the central claim depends on the anomaly-prediction protocol, which is inconsistent across models, and on the accuracy of the reported tables. Those issues must be resolved before the conclusion can be considered supported.
major comments (4)
- [Appendix C.4, D; Tables 2–4] Anomaly prediction is operationalized differently across methods. For TimeGPT, FPT, Time-MOE, and Chronos, the reported prediction metrics are ordinary forecast errors on raw sensor values, converted to anomaly flags only via thresholds described in Appendix D; for MOIRAI, the binary anomaly label column is used directly as the forecasting target (Appendix C.4). Raw forecast error is not equivalent to anomaly-prediction ability, and the baselines (Weighted XGBoost, Autoencoder) are not shown to use the same thresholding protocol. Consequently, the abstract's conclusion that traditional models often match or outperform TSFMs in anomaly prediction is not supported by the evidence as presented. A unified protocol (e.g., fixed thresholds defined on training data only, with precision/recall/F1 on anomaly flags) is needed.
- [Table 4, SMD row] The text states that Weighted XGB consistently yields lower MSE, RMSE, and MAE than the Autoencoder, but the table shows Weighted XGB with MAE 0.93 versus Autoencoder's 0.20, while the RMSE is 0.28 versus 0.54. The MAE value is internally inconsistent with both the adjacent RMSE and the stated claim; this entry must be corrected and the surrounding claims re-verified.
- [Appendix A and Tables 2–4] The MSL dataset is stated to contain 66,709 records, but every MSL row in Tables 2–4 reports #Train = 56K and #Test = 73K, which already sums to 129K and exceeds the total, with no validation split reported. Similar train/test inconsistencies appear elsewhere (e.g., Table 2's FF TimeGPT row has #Train 779K and #Val 194K but no #Test). These discrepancies undermine confidence in the quantitative results and must be rectified.
- [Tables 2 and 3] The claim that fine-tuning yields only marginal improvements is not consistently supported by the tables. For example, on SMD, MOIRAI's MSE worsens from 0.011 (zero-shot) to 1.3 (fine-tuned), and on ETTh1 it is unchanged; the paper does not report variance or significance, so the generalization that fine-tuning gives only marginal improvements is not supported by the presented numbers.
minor comments (5)
- [Table 1 and MOIRAI section] Table 1 classifies MOIRAI as 'Encoder-Only', but the text describes a decoder-only transformer; please align these descriptions.
- [Appendix C.5 and Table 1] Appendix C.5 states that Chronos-tiny has 8M parameters, while Table 1 gives Chronos (tiny) a max model size of 710M; clarify which variant was actually used and correct the inconsistency.
- [Table 2, FF TimeGPT row] The #Test entry is missing for the Future Factories TimeGPT row; include it or explicitly state that no test split was used in that configuration.
- [Analysis Procedure] The statement that 'we replicate the conditions of their original papers' is not supported by a complete list of hyperparameters; provide context lengths, batch sizes, and checkpoints for every model-dataset combination, as some details are only partially reported in Appendix C.
- [References] Several reference entries have formatting issues, for example 'Z ¨ufle' in Zhang et al. 2023; a final proofread of the bibliography is needed.
Circularity Check
No significant circularity: the paper is an empirical benchmark whose anomaly-prediction operationalization is a construct-validity assumption, not a circular derivation.
full rationale
This paper is an empirical evaluation, not a derivation, so the usual circularity patterns do not apply. The central claim that traditional statistical and deep learning models often match or outperform TSFMs in anomaly detection and prediction is supported by held-out test-set comparisons across five public datasets. The authors explicitly disclose that anomaly prediction is operationalized by adapting next-time-series forecasting (footnote 1, Appendix C, Appendix D), and they report ordinary forecast-error metrics (MSE, RMSE, MAE) rather than pretending these metrics are anomaly-classification scores. This is a threat to construct validity, but it is not circularity: the reported numbers are computed from model outputs and ground-truth series independently of the conclusion, and no fitted parameter or threshold is renamed as a prediction. The only self-citation (Shyalika et al. 2024) supplies a definition of anomaly prediction versus detection and is not load-bearing. Appendix D describes 'predefined safe operating thresholds' but does not show that those thresholds were fitted to the test set, so no test-period fit is exhibited. No equation reduces to an input by construction, and no uniqueness theorem or ansatz is imported from the authors' prior work. The main limitations are experimental-design concerns about fairness and metric alignment, not circularity.
Assumptions & free parameters
free parameters (2)
- Feature-specific safe operating thresholds for anomaly prediction
- Per-model feature subsets and context lengths =
e.g., 3 features for TimeGPT on FF, 20 for FPT
assumptions (3)
- domain assumption The chosen datasets and split ratios provide a representative and fair basis for comparing anomaly detection and prediction methods.
- domain assumption Running TSFMs under their original hyperparameters without per-task tuning is a fair zero-shot evaluation.
- ad hoc to paper Forecasting raw series and thresholding the forecast, or forecasting the anomaly label directly, is a valid instantiation of anomaly prediction.
Cite this review
Pith. "Pith review of Time Series Foundational Models: Their Role in Anomaly Detection and Prediction." pith.science (2026). https://pith.science/paper/7AG4IS56
@misc{pith2026241219286,
author = {Pith},
title = {Pith review of: Time Series Foundational Models: Their Role in Anomaly Detection and Prediction},
year = {2026},
howpublished = {\url{https://pith.science/paper/7AG4IS56}},
note = {Machine review of arXiv:2412.19286}
}
read the original abstract
Time series foundational models (TSFM) have gained prominence in time series forecasting, promising state-of-the-art performance across various applications. However, their application in anomaly detection and prediction remains underexplored, with growing concerns regarding their black-box nature, lack of interpretability and applicability. This paper critically evaluates the efficacy of TSFM in anomaly detection and prediction tasks. We systematically analyze TSFM across multiple datasets, including those characterized by the absence of discernible patterns, trends and seasonality. Our analysis shows that while TSFMs can be extended for anomaly detection and prediction, traditional statistical and deep learning models often match or outperform TSFM in these tasks. Additionally, TSFMs require high computational resources but fail to capture sequential dependencies effectively or improve performance in few-shot or zero-shot scenarios. \noindent The preprocessed datasets, codes to reproduce the results and supplementary materials are available at https://github.com/smtmnfg/TSFM.
Figures
Reference graph
Works this paper leans on
-
[1]
L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al
Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774
arXiv 2023
-
[2]
Aggarwal, C. C.; and Aggarwal, C. C. 2017. Outlier ensembles. Springer
work page 2017
-
[3]
F.; Stella, L.; Turkmen, C.; Zhang, X.; Mercado, P.; Shen, H.; Shchur, O.; Rangapuram, S
Ansari, A. F.; Stella, L.; Turkmen, C.; Zhang, X.; Mercado, P.; Shen, H.; Shchur, O.; Rangapuram, S. S.; Arango, S. P.; Kapoor, S.; et al. 2024. Chronos: Learning the language of time series. arXiv preprint arXiv:2403.07815
arXiv 2024
-
[4]
Arjovsky, M.; Bottou, L.; Gulrajani, I.; and Lopez-Paz, D. 2019. Invariant risk minimization. arXiv preprint arXiv:1907.02893
arXiv 2019
-
[5]
Chandola, V.; Banerjee, A.; and Kumar, V. 2009. Anomaly detection: A survey. ACM computing surveys (CSUR), 41(3): 1--58
2009
-
[6]
Dooley, S.; Khurana, G. S.; Mohapatra, C.; Naidu, S. V.; and White, C. 2024. Forecastpfn: Synthetically-trained zero-shot forecasting. Advances in Neural Information Processing Systems, 36
work page 2024
-
[7]
Garza, A.; and Mergenthaler-Canseco, M. 2023. TimeGPT-1. arXiv preprint arXiv:2310.03589
arXiv 2023
-
[8]
Goswami, M.; Szafer, K.; Choudhry, A.; Cai, Y.; Li, S.; and Dubrawski, A. 2024. Moment: A family of open time-series foundation models. In Proceedings of the 41st International Conference on Machine Learning, Vienna, Austria. PMLR 235, 2024
work page 2024
Show all 28 references
-
[9]
E.; Samaha, J.; Clark, D.; Sander, D.; Samaha, P.; Burns, L.; Yousif, I.; Gadow, V.; Tarekegne, T.; et al
Harik, R.; Kalach, F. E.; Samaha, J.; Clark, D.; Sander, D.; Samaha, P.; Burns, L.; Yousif, I.; Gadow, V.; Tarekegne, T.; et al. 2024. Analog and Multi-modal Manufacturing Datasets Acquired on the Future Factories Platform. arXiv preprint arXiv:2401.15544
2024 arXiv
-
[10]
Hundman, K.; Constantinou, V.; Laporte, C.; Colwell, I.; and Soderstrom, T. 2018. Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, 387--395
2018
-
[11]
Y.; Shi, X.; Chen, P.-Y.; Liang, Y.; Li, Y.-F.; Pan, S.; et al
Jin, M.; Wang, S.; Ma, L.; Chu, Z.; Zhang, J. Y.; Shi, X.; Chen, P.-Y.; Liang, Y.; Li, Y.-F.; Pan, S.; et al. 2023. Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728
2023 arXiv
-
[12]
Jin, M.; Zhang, Y.; Chen, W.; Zhang, K.; Liang, Y.; Yang, B.; Wang, J.; Pan, S.; and Wen, Q. 2024. Position: What Can Large Language Models Tell Us about Time Series Analysis. In Forty-first International Conference on Machine Learning
2024
-
[13]
Kaufman, S.; Rosset, S.; Perlich, C.; and Stitelman, O. 2012. Leakage in data mining: Formulation, detection, and avoidance. ACM Transactions on Knowledge Discovery from Data (TKDD), 6(4): 1--21
2012
-
[14]
Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2024. Visual instruction tuning. Advances in neural information processing systems, 36
2024
-
[15]
Liu, X.; McDuff, D.; Kovacs, G.; Galatzer-Levy, I.; Sunshine, J.; Zhan, J.; Poh, M.-Z.; Liao, S.; Di Achille, P.; and Patel, S. 2023. Large language models are few-shot health learners. arXiv preprint arXiv:2305.15525
2023 arXiv
-
[16]
Ranjan, C.; Reddy, M.; Mustonen, M.; Paynabar, K.; and Pourak, K. 2018. Dataset: rare event classification in multivariate time series. arXiv preprint arXiv:1809.10717
2018 arXiv
-
[17]
Rosenblatt, M.; Tejavibulya, L.; Jiang, R.; Noble, S.; and Scheinost, D. 2024. Data leakage inflates prediction performance in connectome-based machine learning models. Nature Communications, 15(1): 1829
2024
-
[18]
Shi, X.; Wang, S.; Nie, Y.; Li, D.; Ye, Z.; Wen, Q.; and Jin, M. 2024. Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts. arXiv preprint arXiv:2409.16040
2024 arXiv
-
[19]
E.; Zi, Y.; Mittal, P.; Narayanan, V.; Harik, R.; and Sheth, A
Shyalika, C.; Roy, K.; Prasad, R.; Kalach, F. E.; Zi, Y.; Mittal, P.; Narayanan, V.; Harik, R.; and Sheth, A. 2024. RI2AP: Robust and Interpretable 2D Anomaly Prediction in Assembly Pipelines. Sensors, 24(10): 3244
2024
-
[20]
Su, Y.; Zhao, Y.; Niu, C.; Liu, R.; Sun, W.; and Pei, D. 2019. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2828--2837
2019
-
[21]
Woo, G.; Liu, C.; Kumar, A.; Xiong, C.; Savarese, S.; and Sahoo, D. 2024. Unified Training of Universal Time Series Forecasting Transformers. arXiv preprint arXiv:2402.02592
2024 arXiv
-
[22]
Wu, H.; Hu, T.; Liu, Y.; Zhou, H.; Wang, J.; and Long, M. 2022. Timesnet: Temporal 2d-variation modeling for general time series analysis. arXiv preprint arXiv:2210.02186
2022 arXiv
-
[23]
Xue, H.; and Salim, F. D. 2023. Promptcast: A new prompt-based learning paradigm for time series forecasting. IEEE Transactions on Knowledge and Data Engineering
2023
-
[24]
R.; Gupta, R
Zhang, X.; Chowdhury, R. R.; Gupta, R. K.; and Shang, J. 2024. Large language models for time series: A survey. arXiv preprint arXiv:2402.01801
2024 arXiv
-
[25]
Zhang, Z.; Amiri, H.; Liu, Z.; Z \"u fle, A.; and Zhao, L. 2023. Large language models for spatial trajectory patterns mining. arXiv preprint arXiv:2310.04942
2023 arXiv
-
[26]
Zhou, T.; Niu, P.; Sun, L.; Jin, R.; et al. 2023. One fits all: Power general time series analysis by pretrained lm. Advances in neural information processing systems, 36: 43322--43355
2023
-
[27]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
-
[28]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.