REVIEW 4 major objections 5 minor 46 references
Towards Foundation Auto-Encoders for Time-Series Anomaly Detection
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper argues that a modest dilated-convolutional variational auto-encoder, pretrained on normal univariate time-series, can track and flag anomalies on previously unseen series in a zero-shot fashion, and that its latent space…
desk verdict Univariate VAE sketch with honest intentions, but the decoder as written cannot produce the time-varying reconstructions shown. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the dilated-convolutional variational auto-encoder: each encoder layer dilates by a factor $F^h$ (with $F$ the filter length and $h$ the layer depth), so the final output at position $T-1$ has seen the whole input $\mathbf{X}$. The number of layers $N$ is the minimum integer satisfying $T \le 2F^{N-1}$, tying architecture size directly to window length. The encoder's two parallel $J$-filter convolutional layers map this output to $\boldsymbol{\mu}_\mathbf{Z}$ and $\boldsymbol{\sigma}_\mathbf{Z}$; the reparameterization trick samples $\mathbf{Z}$, and the decoder, symmetric to the encoder, expands $\mathbf{Z}$ back to $\boldsymbol{\mu}_\mathbf{X}$, $\boldsymbol{\sigma}_\mathbf{X}$. This mechanism is what lets one model observe many series at once and produce an interpretable, reconstruction-based normality region.
What would settle it
Run FAE on the labeled TELCO test set and compute the precision-recall curve obtained by sweeping $\alpha$ in the normality rule; if no single $\alpha$ yields detection performance comparable to a per-series calibrated threshold (or to a simple residual z-score baseline), then the fixed-rule zero-shot claim is falsified. Applying the pretrained model to all 250 KDD2021 series and checking whether its normality regions align with the dataset's known anomalies would reveal whether the 'foundation' behavior extends beyond the 12 series shown.
Extended reading notes
Core claim
FAE adapts the DC-VAE architecture to univariate series: a stack of causal dilated convolutions expands the receptive field exponentially, the encoder compresses the last temporal position into a low-dimensional latent vector $\mathbf{Z}$, and a symmetric decoder repeats that vector across time to output per-timestep parameters $\boldsymbol{\mu}_\mathbf{X}$ and $\boldsymbol{\sigma}_\mathbf{X}$. A window $\mathbf{X}_t$ is normal when it stays within $\alpha$ times the predicted standard deviation of the predicted mean. The paper reports that when the model is trained on several normal series, the latent space organizes by hour of the day, by weekday versus weekend, and by day of the month, and that the decoder tracks seasonality and trends on held-out series; in the zero-shot TS12 experiment, removing TS12 from training leaves predictions almost unchanged as long as a series with similar seasonal and trend behavior is present.
Load-bearing premise
FAE's anomaly detection depends on the rule that a sample is normal only if its deviation from the predicted mean stays within $\alpha$ times the predicted standard deviation; the paper never gives the $\alpha$ used in its figures and provides no quantitative evidence that one fixed $\alpha$ works across heterogeneous series. If this rule needs per-series calibration, the zero-shot detection claim collapses even if the reconstruction tracking is accurate.
Editorial extensions
If this is right
- A single FAE model can monitor all series in a multivariate system one at a time, relying only on univariate temporal structure.
- When the model has seen a temporal pattern in one series, it can track the same pattern zero-shot in a related unseen series, as shown for TS12 after training on TS1 to TS11.
- When the monitored domain shifts from TELCO to TELCO2, the pretrained univariate model keeps tracking, whereas the multivariate DC-VAE requires recalibration.
- Projections of the latent space expose hour-of-day, weekday-versus-weekend, and monthly-trend structure, making the model's normality judgments interpretable.
Reading between the lines
- The architecture's direct tie between window length $T$ and layer count suggests a simple scaling rule for building FAE variants at different temporal resolutions.
- Because the univariate model discards cross-correlations, it may miss anomalies that appear only as spatial deviations; an ensemble that routes between univariate and multivariate detectors could combine both strengths.
- The latent-space 'clock' suggests the VAE encodes the phase of daily seasonality; this phase structure could be exploited for explicit forecasting or for explaining why a point is anomalous.
- The foundation-model claim would be tested more convincingly on the full 250-series KDD2021 set, since the reported experiments use only 12 of the 250 series and do not quantify detection performance on them.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FAE, a VAE with causal dilated convolutional encoder/decoder for univariate time-series modeling, intended as a pretrained 'foundation' model for zero-shot anomaly detection. The authors present qualitative results on the TELCO dataset and on 12 series from the KDD2021 dataset, including a partial zero-shot experiment on TS12, and argue that FAE tracks seasonal and trend behavior and yields interpretable latent representations.
Significance. If the architecture as described is actually able to produce the time-varying reconstructions shown, FAE would be a small-scale, interpretable foundation model for anomaly detection, and the latent-space analysis is a nice demonstration of VAE transparency. The paper also provides a useful departure point for applying VAEs to univariate time series. However, the current manuscript does not provide quantitative validation, and the architectural description contains a serious internal inconsistency; the significance of the contribution therefore cannot be assessed as it stands.
major comments (4)
- [Section 3, Figure 1, Figures 3, 9, 10] The decoder as described cannot produce time-varying reconstructions. The input to the decoder is the latent vector Z repeated T times, forming a constant sequence along the time axis. Since all decoder layers are causal dilated convolutions, a constant input produces a constant output for all time steps (except for boundary effects). Yet Figures 3, 9, and 10 show strongly time-varying μ_X and σ_X that track daily seasonality and trends. This is a load-bearing discrepancy: without a mechanism that breaks the temporal symmetry (e.g., per-timestep conditioning, a non-constant latent input, or autoregressive connections), the reported results cannot be reproduced by the architecture. The authors must either provide the actual decoder mechanism or correct the description.
- [Section 3 (anomaly rule) and Section 4] The paper defines the anomaly detection rule using a threshold α—a sample is normal only if its deviation from μ_X stays within α times σ_X—but never reports the α value used in Figures 7 and 8, and provides no quantitative anomaly detection results (e.g., precision, recall, F1, or ROC). The evaluation is entirely qualitative, based on visual inspection, with no error bars and no baselines beyond the earlier DC-VAE, which is also not quantified. Without a specified threshold and quantitative metrics, the claim that FAE detects anomalies on unseen data is unsupported.
- [Section 4.5] The zero-shot experiment for TS12 is confounded by the high correlation between TS12 and TS11: training on TS1–TS11 gives the model direct evidence of TS12's seasonal and trend patterns, so this is not a clean test of zero-shot generalization. The more challenging setting (training on TS1–TS10) shows degraded performance, but is only evaluated qualitatively. To substantiate zero-shot generalization, the authors need quantitative comparisons across multiple held-out series and domains, with baselines and statistical significance.
- [Section 4.6] The KDD2021-12 experiment trains on the train partition and tests on the test partition of the same 12 series, which is within-series transfer, not zero-shot or foundation-model generalization. The paper's stated goal is to model 'previously unseen datasets,' yet the only partially held-out evaluation is the TS12 experiment in TELCO. A proper foundation-model evaluation should pretrain on a large, diverse corpus and test on completely unseen series from different domains without any retraining, which is not done here.
minor comments (5)
- [Section 3] The description of the receptive field expansion ('the dilation of each layer at the encoder increases exponentially as the network deepens') is imprecise; consider giving an explicit formula, e.g., receptive field size after N layers with kernel F and base dilation d.
- [Figure 4] The abbreviation 'ZPC' is used for the principal components of the latent vectors, but it is not defined in the caption or text; please write out 'principal component of Z' or explain the notation.
- [Abstract and Section 1] The abstract mentions 'preliminary results in different multi-dimensional time-series datasets,' but FAE is a univariate model; please clarify that the datasets are multivariate but the model analyzes each series separately, or rephrase to avoid confusion.
- [References] Reference [22] for the KDD2021 dataset points to a competition URL; a proper citation to the dataset itself (e.g., the UCR archive or the competition organizers' publication) would be more appropriate.
- [Figures 7 and 8] The captions mention 'flagged anomalies,' but the anomaly markers and the threshold used to flag them are not described in the captions or the main text; adding a legend and the α value would greatly improve interpretability.
Circularity Check
No significant circularity: FAE's held-out and zero-shot evaluations do not reduce to fitted inputs or self-citation chains.
full rationale
The paper's central claims are empirical: FAE, a VAE with dilated convolutional encoder/decoder, is pretrained on TELCO series and evaluated on held-out TELCO testing samples, TELCO2, and KDD2021-12. The zero-shot experiment with TS12 is a genuine holdout: the model is trained on TS1-TS11 or TS1-TS10 and then applied to TS12 (Section 4.5). No parameter is fitted to the target series; the alpha threshold rule in Section 3 is a decision rule, not a fitted quantity, and although alpha is not reported, that is a completeness/correctness gap rather than circularity. The architecture is taken from the authors' prior DC-VAE work [12,13], but this self-citation is provenance, not a load-bearing uniqueness or optimality argument; the experiments here independently test the resulting model on unseen data. The KDD2021 evaluation trains on the provided train partition and reports on test partition (Section 4.6), which is standard. The discussion honestly states limitations (small number of datasets/domains, Section 5). The skeptic's concern that a decoder fed a time-repeated latent vector cannot produce time-varying output is a mechanistic correctness question about the described architecture; it does not exhibit a circular equivalence between reported predictions and inputs, so it is outside the circularity score.
Assumptions & free parameters
free parameters (6)
- T (sequence length) =
256
- J (latent dimension) =
48
- gamma (learning rate) =
6e-5
- m (mini-batch size) =
32
- U (number of filters) =
128
- alpha (anomaly threshold) =
unreported
assumptions (3)
- domain assumption The VAE latent space is a centered isotropic Gaussian with approximately diagonal-covariance posteriors.
- domain assumption A fixed integer alpha times the predicted standard deviation defines the normal region for every time-series.
- standard math Causal dilated convolutional layers with exponential dilation give each output time an effective receptive field covering the whole window.
Cite this review
Pith. "Pith review of Towards Foundation Auto-Encoders for Time-Series Anomaly Detection." pith.science (2026). https://pith.science/paper/VV3EAOVK
@misc{pith2026250701875,
author = {Pith},
title = {Pith review of: Towards Foundation Auto-Encoders for Time-Series Anomaly Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/VV3EAOVK}},
note = {Machine review of arXiv:2507.01875}
}
read the original abstract
We investigate a novel approach to time-series modeling, inspired by the successes of large pretrained foundation models. We introduce FAE (Foundation Auto-Encoders), a foundation generative-AI model for anomaly detection in time-series data, based on Variational Auto-Encoders (VAEs). By foundation, we mean a model pretrained on massive amounts of time-series data which can learn complex temporal patterns useful for accurate modeling, forecasting, and detection of anomalies on previously unseen datasets. FAE leverages VAEs and Dilated Convolutional Neural Networks (DCNNs) to build a generic model for univariate time-series modeling, which could eventually perform properly in out-of-the-box, zero-shot anomaly detection applications. We introduce the main concepts of FAE, and present preliminary results in different multi-dimensional time-series datasets from various domains, including a real dataset from an operational mobile ISP, and the well known KDD 2021 Anomaly Detection dataset.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
Justin Bayer and Christian Osendorfer. 2014. Learning stochastic recurrent networks. arXiv preprint arXiv:1411.7610 (2014)
arXiv 2014
-
[2]
James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011. Algorithms for Hyper-parameter Optimization. Advances in neural information processing systems 24 (2011)
work page 2011
-
[3]
Dalca, Luca Saglietti, Jennifer Listgarten, and Nicoló Fusi
Francesco Paolo Casale, Adrian V. Dalca, Luca Saglietti, Jennifer Listgarten, and Nicoló Fusi. 2018. Gaussian Process Prior Variational Autoencoders. In Advances in Neural Information Processing Systems
work page 2018
-
[4]
Chen, Lee Dicker, Carson Eisenach, and Dhruv Madeka
Kevin C. Chen, Lee Dicker, Carson Eisenach, and Dhruv Madeka. 2022. MQTrans- former: Multi-horizon Forecasts with Context Dependent Attention and Optimal Bregman Volatility. InKDD 2022 Workshop on Mining and Learning from Time Series – Deep Forecasting: Models, Interpretability, and Applications
work page 2022
-
[5]
Run-Qing Chen, Guang-Hui Shi, Wanlei Zhao, and Chang-Hui Liang. 2021. A joint model for IT operation series prediction and anomaly detection. Neurocomputing 448 (2021), 130–139
work page 2021
-
[6]
Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Bengio. 2015. A recurrent latent variable model for sequential data. Advances in Neural Information Processing Systems 28 (2015)
work page 2015
-
[7]
Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. 2024. A Decoder-only Foundation Model for Time-series Forecasting. arXiv:2310.10688 [cs.CL]
arXiv 2024
-
[8]
Carl Doersch. 2016. Tutorial on variational autoencoders. arXiv preprint arXiv:1606.05908 (2016)
arXiv 2016
Show all 46 references
-
[9]
Jeff Donahue, Philipp Krähenbühl, and Trevor Darrell. 2016. Adversarial feature learning. arXiv preprint arXiv:1605.09782 (2016)
2016 arXiv
-
[10]
Vincent Fortuin, Dmitry Baranchuk, Gunnar Rätsch, and Stephan Mandt. 2020. GP-VAE: Deep Probabilistic Time Series Imputation. InInternational conference on artificial intelligence and statistics . PMLR, 1651–1661
2020
-
[11]
Gastón García González, Pedro Casas, Alicia Fernández, and Gabriel Gómez
-
[12]
Gastón García González, Sergio Martinez Tagliafico, Alicia Fernández, Gabriel Gómez, José Acuña, and Pedro Casas. 2022. DC-VAE, Fine-grained Anomaly Detection in Multivariate Time-Series with Dilated Convolutions and Varia- tional Auto Encoders. In 2022 IEEE European Symposium...
2022
-
[13]
Gastón García González, Sergio Martinez Tagliafico, Alicia Fernández, Gabriel Gómez, José Acuña, and Pedro Casas. 2023. One Model to Find Them All – Deep Learning for Multivariate Time-Series Anomaly Detection in Mobile Network Data. IEEE Transactions on Network and Service Ma...
2023
-
[14]
Gastón García González, Pedro Casas, and Alicia Fernández. 2023. Fake it till you Detect it: Continual Anomaly Detection in Multivariate Time-Series using Gen- erative AI. In 2023 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW). 558–566. https://doi.org/10...
2023
-
[15]
Gastón García González, Sergio Martínez Tagliafico, Alicia Fernández, Gabriel Gómez, José Acuña, and Pedro Casas. 2023. TELCO – a new Multivariate Time- Series Dataset for Anomaly Detection in Mobile Networks. https://doi.org/10. 21227/skpg-0539
2023
-
[16]
Azul Garza and Max Mergenthaler-Canseco. 2023. TimeGPT-1. arXiv:2310.03589 [cs.LG]
2023 arXiv
-
[17]
Alexander Geiger, Dongyu Liu, Sarah Alnegheimish, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. 2020. TadGAN: Time series anomaly detection using generative adversarial networks. In 2020 IEEE International Conference on Big Data (Big Data). IEEE, 33–43
2020
-
[18]
Laurent Girin, Fanny Roche, Thomas Hueber, and Simon Leglaive. 2019. Notes on the use of variational autoencoders for speech and audio spectrogram modeling. In DAFx 2019-22nd International Conference on Digital Audio Effects . 1–8
2019
-
[19]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems , Vol. 27
2014
-
[20]
Nate Gruver, Marc Finzi, Shikai Qiu, and Andrew Gordon Wilson. 2023. Large Lan- guage Models Are Zero-Shot Time Series Forecasters. arXiv:2310.07820 [cs.LG]
2023 arXiv
-
[21]
Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, and Qingsong Wen. 2024. Time-LLM: Time series forecasting by reprogramming large language models. In International Conference on Learning Representat...
2024
-
[22]
Keogh, T
E. Keogh, T. Dutta Roy, U. Naik, and A. Agrawal. 2021. Multi-dataset Time-Series Anomaly Detection Competition, SIGKDD 2021. https://compete.hexagon- ml.com/practice/competition/39/
2021
-
[23]
Diederik P Kingma and Max Welling. 2013. Auto-encoding Variational Bayes. CoRR abs/1312.6114 (2013). arXiv:1312.6114 https://arxiv.org/abs/1312.6114
2013 arXiv
-
[24]
Diederik P Kingma and Max Welling. 2019. An introduction to variational autoencoders. arXiv preprint arXiv:1906.02691 (2019)
2019 arXiv
-
[25]
Guokun Lai, Bohan Li, Guoqing Zheng, and Yiming Yang. 2018. Stochastic WaveNet: A Generative Latent Variable Model for Sequential Data.arXiv preprint arXiv:1806.06116 (2018)
2018 arXiv
-
[26]
Dan Li, Dacheng Chen, Baihong Jin, Lei Shi, Jonathan Goh, and See-Kiong Ng
-
[27]
Bryan Lim, Sercan Arik, Nicolas Loeff, and Tomas Pfister. 2021. Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting. Interna- tional Journal of Forecasting 37, 4 (2021), 1748–1764. 8 Towards Foundation Auto-Encoders for Time-Series Anomaly Dete...
2021
-
[28]
Guansong Pang, Chunhua Shen, Longbing Cao, and Anton Van Den Hengel
-
[29]
Siddharth Ramchandran, Gleb Tikhonov, Kalle Kujanpää, Miika Koskinen, and Harri Lähdesmäki. 2021. Longitudinal variational autoencoder. In International Conference on Artificial Intelligence and Statistics . PMLR, 3898–3906
2021
-
[30]
Kashif Rasul, Arjun Ashok, Andrew Robert Williams, Hena Ghonia, Rishika Bhag- watkar, Arian Khorasani, Mohammad Javad Darvishi Bayazi, George Adamopou- los, Roland Riachi, Nadhir Hassen, Marin Biloš, Sahil Garg, Anderson Schneider, Nicolas Chapados, Alexandre Drouin, Valentina...
2024 arXiv
-
[31]
ACM Comput
Deep Learning for Anomaly Detection: A Review. ACM Comput. Surv. 54, 2, Article 38 (March 2021), 38 pages
2021
-
[32]
Samira Shabanian, Devansh Arpit, Adam Trischler, and Yoshua Bengio. 2017. Variational bi-LSTMs. arXiv preprint arXiv:1711.05717 (2017)
2017 arXiv
-
[33]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems , Vol. 30
2017
-
[34]
Sebastian Schmidl, Phillip Wenig, and Thorsten Papenbrock. 2022. Anomaly detection in Time Series: a Comprehensive Evaluation. Proc. VLDB Endow. 15, 9 (may 2022), 1779–1797. https://doi.org/10.14778/3538598.3538602
2022
-
[35]
Hao Xue and Flora D. Salim. 2023. PromptCast: A New Prompt-Based Learning Paradigm for Time Series Forecasting. IEEE Transactions on Knowledge and Data Engineering (2023), 1–14. https://doi.org/10.1109/TKDE.2023.3342137
2023
-
[36]
Zichao Yang, Zhiting Hu, Ruslan Salakhutdinov, and Taylor Berg-Kirkpatrick
-
[37]
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. 2021. Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Fore- casting. CoRR abs/2106.13008 (2021). arXiv:2106.13008 https://arxiv.org/abs/ 2106.13008
2021 arXiv
-
[38]
Sultan Zavrak and Murat Iskefiyeli. 2020. Anomaly-based intrusion detection from network flow features using variational autoencoder. IEEE Access 8 (2020), 108346–108358
2020
-
[39]
Houssam Zenati, Chuan Sheng Foo, Bruno Lecouat, Gaurav Manek, and Vijay Ra- maseshan Chandrasekhar. 2018. Efficient GAN-based anomaly detection. arXiv preprint arXiv:1802.06222 (2018)
2018 arXiv
-
[40]
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. In Thirty-Fifth AAAI Conference on Artificial Intelligence. AAAI Press, 11106–11115. https://d...
2021 doi
-
[41]
Jinsung Yoon, Daniel Jarrett, and Mihaela van der Schaar. 2019. Time-series Generative Adversarial Networks. In Advances in Neural Information Processing Systems, Vol. 32
2019
-
[45]
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin
-
[2017]
In International conference on machine learning
Improved Variational Autoencoders for Text Modeling using Dilated Con- volutions. In International conference on machine learning . PMLR, 3881–3890
-
[2019]
In International Conference on Artificial Neural Networks
MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks. In International Conference on Artificial Neural Networks. Springer, 703–716
-
[2021]
SIGMETRICS Perform
On the Usage of Generative Models for Network Anomaly Detection in Multivariate Time-Series. SIGMETRICS Perform. Eval. Rev. 48, 4 (may 2021), 49–52. https://doi.org/10.1145/3466826.3466843
2021
-
[2022]
CoRR abs/2201.12740 (2022)
FEDformer: Frequency Enhanced Decomposed Transformer for Long- term Series Forecasting. CoRR abs/2201.12740 (2022). arXiv:2201.12740 https: //arxiv.org/abs/2201.12740 9
2022 arXiv
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.