{"id":"5fba5214-73d6-43b0-ac13-718b64f7e51b","arxiv_id":"2501.07814","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"STTS-EAD embeds anomaly detection into the training loop of a spatio-temporal forecaster and reports 3.8 to 8.1 percent RMSE gains over baselines.","lead":"STTS-EAD is a machine learning method that detects and smooths anomalies in training data while training a multivariate time series forecaster, instead of cleaning data as a separate step. On two coffee sales datasets and an S&P 500 stock dataset, it reports lower prediction errors than nine baseline forecasting models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"EAD's gains are not tied to genuine anomaly detection: residual-based self-cleaning may just be smoothing, and the paper never validates detection against ground truth.","rationale":"The reader's weakest assumption is exactly the reliability of residual-based anomaly detection in the self-referential cleaning loop. My stress test confirms this is the load-bearing concern: the paper's central empirical claim depends on EAD distinguishing anomalies from noise, yet there is no direct evidence that it does. The numerical inconsistency between Table II Part 1 and Part 3 (1.09157 vs 1.09557 for the same STTS-EAD setting on Coffee-Cream) further undermines confidence in the reported gains, since the difference is of the same order as the claimed improvement. However, the paper does provide some supporting evidence: the EAD module outperforms alternative preprocessing approaches, and ablation studies show each architectural component contributes. These make the approach plausible, so a conditional verdict remains appropriate. The proposed synthetic-anomaly experiment would directly test the causal role of anomaly detection and settle whether the central claim holds. My recommendation is therefore to keep the reader's CONDITIONAL verdict unchanged, with the added condition that the authors validate the anomaly detection mechanism against ground truth and report variance across seeds.","tokens_in":14134,"tokens_out":5428,"duration_ms":56106,"concrete_test":"Run a controlled experiment on synthetic multivariate time series with known injected anomalies. Compare STTS-EAD against two controls: (1) STTS trained with the same smoothing replacement applied to the same number of points selected at random, and (2) STTS trained on original uncleaned data. Use at least five seeds and report mean and standard deviation of RMSE. If STTS-EAD does not outperform the random-cleaning control by more than the run-to-run variance, the residual-based detection is not the source of the gains. Additionally, report precision and recall of EAD-detected anomalies against the injected ground truth to verify that the module actually detects anomalies.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that embedding anomaly detection into training improves forecasting because the model detects and corrects true anomalies using spatiotemporal residuals. This requires the residual scores from the current model to be reliable indicators of anomalies. The paper never validates that assumption: no ground-truth anomaly labels, no synthetic injection experiments, and no precision/recall analysis. Because the model labels its own training data and then trains on the cleaned data, a feedback loop exists: 'anomalies' are just high-residual points under the current model, which may be noise or hard-to-fit samples. The observed gains could therefore come from the smoothing operation acting as a regularizer rather than from removing genuine anomalies. This is compounded by a numerical inconsistency: Table II Part 1 reports STTS-EAD Coffee-Cream RMSE 1.09157, while Part 3 reports the same model as 1.09557. This run-to-run variability is comparable to the claimed 3.8% improvement, weakening the significance of the result without error bars. The paper's ablation against other preprocessing methods (Part 3) does not resolve this, because those baselines also smooth data; the comparison shows only that model-based cleaning helps more, not that the cleaned points are truly anomalous. If the EAD advantage vanishes when matched-amount random smoothing is applied, the core contribution is not anomaly detection but generic smoothing.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes STTS-EAD, a multivariate time series forecasting method that embeds an anomaly detection module into the training loop. The STTS model learns temporal and spatial embeddings, selects auxiliary series, and combines attention, transformer, and LSTM components for prediction and reconstruction. The EAD module periodically computes prediction and reconstruction residuals, labels high-residual points as anomalies using a nonparametric dynamic threshold, replaces them with smoothed values, and retrains the model on the cleaned data. Experiments on two proprietary coffee sales datasets and the public Stock-SP500 dataset report RMSE/MAE improvements over nine baselines, with claimed RMSE gains of 5.6%, 3.8%, and 8.1% over the best baselines on the three datasets. The paper also includes ablations of the STTS components and a comparison of EAD with classical and deep anomaly-preprocessing methods.","tokens_in":14337,"tokens_out":4162,"duration_ms":43282,"significance":"If the mechanism holds, the central idea of coupling anomaly-driven data correction with forecasting training in a spatio-temporal model is of practical interest, and the dynamic adaptation to a varying number of time series is a useful property. The paper's strength is the comprehensive set of baselines and ablations, and the explicit comparison of EAD with two-stage preprocessing alternatives. However, the verification is limited: the two primary datasets are proprietary, the reported gains are not accompanied by error bars or significance tests, and the anomaly detection component is not validated against ground truth. The result is therefore promising but not yet convincing as stated.","major_comments":[{"comment":"The numerical results for the same model are inconsistent: STTS-EAD Coffee-Cream RMSE is reported as 1.09157 and MAE 0.74425 in Part 1, but as 1.09557 and 0.74919 in Part 3. With the STTS baseline at 1.11719, the claimed 3.8% gain in Part 1 becomes approximately 1.9% if the Part 3 value is used. This discrepancy is comparable in magnitude to the reported improvement and must be reconciled, and the paper should report means and standard deviations over multiple random seeds to establish that the gains are not run-to-run artifacts.","section":"Table II (Parts 1 and 3), Section IV.B and IV.D.1"},{"comment":"The EAD module labels training points as anomalous using the current model's own prediction and reconstruction residuals, then replaces them and retrains. The paper never validates these labels: no ground-truth anomaly annotations, no synthetic anomaly injection, and no precision/recall or F1 analysis. Consequently, the reported gains could stem from generic smoothing of high-residual (possibly hard-to-fit or noisy) points rather than from genuinely detecting anomalies. The ablation in Part 3 of Table II compares against other smoothing-based preprocessing methods, but it does not include a control that smooths the same number of randomly selected points. Such a control (or a synthetic injection study) is necessary to support the claim that EAD is performing anomaly detection rather than a generic regularizer.","section":"Section III.C, Algorithm 1, and Section IV.D"},{"comment":"The paper states that STTS-EAD 'significantly outperform[s] baselines,' but no statistical significance tests or error bars are provided. The improvement over STTS on Stock-SP500 is only 0.65% in RMSE (0.01231 vs 0.01223), and the Coffee-Cream inconsistency noted above shows that variability can be of the same order as the reported gains. The authors should report means and standard deviations across multiple runs and, where appropriate, a paired significance test against the strongest baseline.","section":"Section IV.B, Table II"}],"minor_comments":[{"comment":"The arrows (↑ and ↓) in Part 3 are not defined in the caption, and their direction is unintuitive (a lower RMSE is marked ↑). Please clarify the notation or use a different marker.","section":"Table II, Part 3"},{"comment":"The notation {ε^p_1, ..., ε^p_{N×T}} is ambiguous: are these errors defined per time series and per timestamp, and if so how are they aligned with the sliding-window inputs used by the model? Please define the indexing explicitly.","section":"Algorithm 1, line 5"},{"comment":"The 'nonparametric dynamic thresholding method [30]' is invoked without describing how it is applied to the anomaly scores. A brief description of the threshold computation (e.g., the smoothing and extreme-value steps from Hundman et al.) would improve reproducibility.","section":"Section III.C"},{"comment":"The claim that STTS-EAD is 'the first to integrate anomaly detection and forecasting tasks in the training phase' is a strong novelty claim that is not supported by a dedicated survey of training-time cleaning or robust learning approaches. Please narrow the claim to the specific residual-based self-cleaning framework proposed here, or add a more thorough comparison with prior joint training schemes.","section":"Abstract and Section I"},{"comment":"The caption 'Performance with different filling Data. (RMSE)' should state that lower values are better and specify that the methods compared are 'mean', 'period mean', 'lowess', and 'remove'.","section":"Section IV.D.3, Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The reviewer's main concerns are the internal inconsistency in the central results table, the lack of validation for the anomaly detection component, and the absence of error bars. The proprietary coffee data make independent replication impossible, which raises the bar for the authors to provide code and statistical evidence. The paper is technically coherent but overclaims in both the 'first' novelty statement and the significance of the numerical improvements."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is worth a look: clean the training data inside the training loop using the model's own residuals, rather than as a separate preprocessing step. That specific integration is new as far as I know. The STTS base model is a standard bag of tricks—attention, GCN, transformer, LSTM—but the auxiliary series selection via spatiotemporal embeddings is a nice touch for datasets where the number of series changes over time. The paper also runs a thorough component ablation, and the comparison with four alternative preprocessing methods is a sensible way to position the EAD module. If the numbers are right, the gains are consistent across three datasets, though modest (1.6–8.1% RMSE).\n\nThe soft spots are real and load-bearing. No code, no error bars, no significance tests, and two of the three datasets are proprietary. The numerical inconsistency in Table II—Coffee-Cream RMSE for STTS-EAD is 1.09157 in Part 1 but 1.09557 in Part 3—needs an explanation. More importantly, the EAD module labels anomalies based on the model's own prediction and reconstruction errors, then trains on the cleaned data. That loop is self-referential. The paper never validates that the labeled points are true anomalies, so the observed gains could come from smoothing that acts as a regularizer, not from removing genuine anomalies. The comparison with 3σ, EWMA, USAD, and LSTM-NDT does not resolve this, because those methods also smooth; it only shows that model-based cleaning helps more. An experiment with matched-amount random smoothing would clarify whether the \"anomaly detection\" part is doing the work.\n\nThe 'first to integrate' claim is also a little strong. MTAD-GAT does jointly optimize prediction and reconstruction for anomaly detection; not the same as cleaning training data for forecasting, but it undercuts the absolute novelty.\n\nWho would benefit? Someone working on MTS forecasting with noisy data, or anyone interested in self-training pipelines. It's not a breakthrough, but the EAD loop is a transferable idea. I would send it to review, expecting major revision: fix the inconsistency, add statistical rigor, validate the anomaly detection against ground truth or a random-smoothing control, and tone down the novelty claim. As it stands, I would not cite it in the next year, but I'd keep an eye on the authors' future work.","headline":"Plausible idea, but thin evidence and a self-referential cleaning loop make the strong claims hard to trust—worth a serious referee, not a pass.","tokens_in":14925,"tokens_out":4540,"would_cite":false,"duration_ms":43304,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"STTS-EAD embeds anomaly detection into the training loop of a spatio-temporal forecasting model and reports consistent RMSE reductions of 5.6%, 3.8%, and 8.1% over the best baselines on three datasets.","keywords":["multivariate time series forecasting","anomaly detection","spatio-temporal learning","embedded anomaly detection","data cleaning","attention mechanism","sales forecasting","stock price prediction"],"falsifier":"Run STTS-EAD on synthetic multivariate time series with known injected anomalies of varying size and record whether the flagged positions match the injections and whether cleaning recovers the true signal; if flagged sets are mostly noise or if equally many random points smoothed produce the same RMSE gains, the reported improvements do not come from anomaly detection.","tokens_in":13877,"feed_emoji":"📈","tokens_out":5496,"duration_ms":51248,"temperature":0.7,"pith_summary":"The paper argues that anomaly handling for multivariate time series forecasters should not be a separate preprocessing stage: the forecasting model itself can score its own training points during training, flag likely anomalies, and replace them with smoothed values, and alternating this cleaning with model updates improves forecast accuracy. To make the case, it introduces STTS-EAD, a spatio-temporal forecasting model with an embedded anomaly-detection module, and reports RMSE improvements of 5.6%, 3.8%, and 8.1% over the strongest baseline on coffee-sales and S&P 500 datasets. If correct, the result turns data cleaning from an offline statistical chore into a trainable component that can use the same spatio-temporal context as the predictor.","feed_headline":"Training-time anomaly cleaning cuts forecast error by up to 8.1%","feed_subtitle":"Embedding anomaly detection in the training loop beats two-stage cleaning on sales and stock datasets.","key_machinery":"The load-bearing mechanism is a residual-score feedback loop the paper calls the Embedded Anomaly Detection (EAD) module. Every $\\eta$ training epochs the module computes, for each training point, an anomaly score $s_i = \\delta \\epsilon^p_i + (1-\\delta) \\epsilon^r_i$ from the current model's prediction error and reconstruction error, sets a threshold with nonparametric dynamic thresholding, and replaces points above the threshold with smoothed filler data. The cleaned set is then used to continue training the STTS forecaster, which itself supplies the residuals, so detection and forecasting are repeatedly optimized against each other. Supporting machinery includes concatenated temporal and spatial embeddings (with a GCN when a prior graph exists), a correlation-based auxiliary series selection that keeps the input dimension fixed, and temporal/spatial attention plus a feature-wise transformer and LSTM that produce the prediction and reconstruction outputs.","core_discovery":"The paper's central claim is that integrating anomaly detection into the training phase, rather than running it as a separate preprocessing step, improves multivariate time series forecasting accuracy. STTS-EAD alternates between optimizing a spatio-temporal prediction model and running an Embedded Anomaly Detection module every few epochs: the module takes prediction and reconstruction residuals from the current model, combines them into a weighted anomaly score, thresholds the scores with a nonparametric dynamic method, and replaces flagged training points with smooth filler data. The cleaned dataset then feeds the next round of training. The paper reports that this procedure lowers RMSE by 5.6% on Coffee-Bean, 3.8% on Coffee-Cream, and 8.1% on Stock-SP500 relative to the best baseline, and that the same embedding of anomaly detection beats three statistical and deep-learning preprocessing alternatives when each is attached to the same STTS base model.","pith_inferences":["A direct extension, not tested in the paper: the same EAD loop could be bolted onto other forecasters that already produce prediction and reconstruction errors, without the STTS architecture, to test whether the cleaning mechanism transfers.","Because the model labels its own training data, early epochs may flag ordinary noise or hard-but-informative points; a warm-up schedule that activates EAD later, or that trusts scores only after validation stabilizes, is a natural robustness check.","The stock dataset has no prior graph, yet STTS-EAD still improves, suggesting learned embeddings can substitute for explicit spatial structure; injecting controlled anomalies into synthetic data would let one measure detection precision directly.","The reported gains may partly reflect regularization from smoothing rather than accurate anomaly identification; comparing against random smoothing of the same number of points would separate those effects."],"forward_implications":["If the reported gains hold, two-stage pipelines that clean data before training should be revisited: cleaning inside the training loop can outperform 3-sigma, EWMA, USAD, and LSTM-NDT preprocessing on the same base forecaster.","Combining prediction and reconstruction residuals in the anomaly score performs better than either residual alone across the three datasets, so models that produce both outputs are natural hosts for this style of embedded cleaning.","The auxiliary series selection module lets the model handle a changing number of time series without retraining, because the input dimension is fixed by the selected M series.","Every ablated STTS component—auxiliary selection, spatial attention, temporal attention, feature-wise transformer, and LSTM—increases error, supporting the claim that spatio-temporal structure drives the gains."],"supporting_citations":[{"why":"Supplies the nonparametric dynamic thresholding used by the EAD module to separate anomalous points from normal ones.","marker":"[30]"},{"why":"Provides the GCN used to encode prior graph information into spatial embeddings in STTS.","marker":"[49]"},{"why":"Provides the GATv2 attention mechanism used for temporal and spatial feature learning in STTS.","marker":"[50]"},{"why":"MTGNN is the strongest graph-based baseline and the main comparison point for the claimed gains.","marker":"[25]"},{"why":"DLinear and NLinear are the modern MLP baselines that the method must beat.","marker":"[21]"},{"why":"Autoformer is a transformer-based baseline used in the comparison and ablation experiments.","marker":"[16]"}],"fun_headline_variants":["Embed anomaly detection into training to cut forecast errors","One loop for anomalies and forecasts: STTS-EAD","In-training anomaly detection lifts time series accuracy","Up to 8.1% lower RMSE by embedding anomaly detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The training loop assumes the model's own prediction and reconstruction errors reliably mark the truly anomalous points, and that replacing those points with smoothed values improves the data, even though no ground-truth anomaly labels are used to verify either step.","fun_headline_variants_meta":{"raw":{"variants":["Embed anomaly detection into training to cut forecast errors","One loop for anomalies and forecasts: STTS-EAD","In-training anomaly detection lifts time series accuracy","Up to 8.1% lower RMSE by embedding anomaly detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000185,"raw_usage":{"total_tokens":1323,"prompt_tokens":947,"completion_tokens":376,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":563,"completion_tokens_details":{"reasoning_tokens":311}},"tokens_in":563,"tokens_out":376,"duration_ms":4114,"temperature":1.0,"reasoning_tokens":311,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:34:37.589986+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run STTS-EAD on synthetic multivariate time series with known injected anomalies of varying size and record whether the flagged positions match the injections and whether cleaning recovers the true signal; if flagged sets are mostly noise or if equally many random points smoothed produce the same RMSE gains, the reported improvements do not come from anomaly detection.","supporting_citations":[{"cited_title":"Con- necting the dots: Multivariate time series forecasting with graph neural networks,","cited_arxiv_id":null,"evidence_quote":"MTGNN is the strongest graph-based baseline and the main comparison point for the claimed gains."}],"review_version":1}