{"id":"3087e4b4-8296-4ff2-89f6-bdeac40d64e0","arxiv_id":"2412.02474","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A frequency-matrix plus SENet-LSTM pipeline is proposed for time-series anomaly detection and reported to beat five baselines, but the comparison is compromised by overlapping-window data leakage.","lead":"F-SE-LSTM is a neural-network method that converts chunks of a time series into frequency patterns and then uses attention and memory layers to spot anomalies. The authors report top F1 scores on Yahoo and Numenta benchmark data, but the evaluation leaks information between training and test windows, so the claimed advantage is not established.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation is invalid because overlapping sliding-window samples are randomly split into train/validation/test, so every test window has near-duplicate windows in training and reported F1 does not measure out-of-sample anomaly detection.","rationale":"The strongest claim in the paper is empirical: F-SE-LSTM outperforms existing state-of-the-art anomaly detection methods in capability and efficiency. The entire support for this claim is the comparison in Section 4.4.5 and Table 8. The load-bearing assumption is that the test set measures generalization to unseen anomalous patterns. Section 3.1 constructs overlapping length-60 windows with M = L - N + 1, so neighboring windows share 59 of 60 raw observations, and Section 4.2 randomly splits those windows into training, validation, and test sets. This destroys the independence between partitions. With sample-level labels defined by 'contains an outlier,' a model can achieve high F1 by memorizing near-duplicate windows rather than by learning a general frequency-matrix representation. This is not a minor statistical imperfection; it directly undermines the paper's headline contribution. I agree with the reader's weakest-assumption analysis. One nuance is worth stating honestly: the baselines are evaluated under the same leaky protocol, so a relative ranking could in principle survive, but the paper claims capability on the datasets themselves, and no valid out-of-sample comparison is made. The absolute F1 margins, roughly 1.2% over the best baseline, are within the range that leakage artifacts can plausibly produce. Secondary weaknesses, including choosing T by test-set F1 and reporting no error bars, reinforce the need for re-evaluation. The architecture itself is a reasonable composition of known components, and the frequency-matrix representation could be a legitimate research direction, but the empirical evidence as presented does not support the SOTA claim. The reader's REJECT verdict is therefore appropriate, and my stress-test does not change it.","tokens_in":15151,"tokens_out":4842,"duration_ms":53175,"concrete_test":"Use the released GitHub code to rerun the comparison under a contamination-free protocol: split each raw series into consecutive time blocks before windowing, for example assigning windows whose start index lies in the first 60% of each series to training, the next 20% to validation, and the last 20% to testing, so no raw observation appears in more than one partition. Recompute the average F1, recall, precision, and accuracy in Table 8 for F-SE-LSTM and all five baselines, with at least five random seeds and 95% confidence intervals. Also rerun the choice of T on the validation set only. If F-SE-LSTM's margin over the best baseline shrinks below the reported 1.2% average F1 or overlaps the baselines' uncertainty, the outperformance claim fails. If the margin persists, the claim needs to be restated under the corrected protocol.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Eq. (1), M = L - N + 1 with N = 60, so consecutive samples share 59 of 60 raw observations and every raw point appears in up to 60 samples. Section 4.2 then says random stratified sampling divides the dataset into 60/20/20 train/validation/test sets. This is not independent evaluation: a test window [x_i, ..., x_{i+59}] has training windows [x_{i-1}, ..., x_{i+58}], [x_{i-2}, ..., x_{i+57}], etc., that overlap it by 59 points, and labels are defined at the sample level as 'contains an outlier.' The classifier can memorize near-duplicate windows and their labels instead of detecting anomalies. The reported F1 scores (0.9799 on A1, 0.9728 on AWS, 0.9256 on Known, 0.9969 on Traffic; average 0.9688) therefore reflect leakage, not out-of-sample capability. Because the same leaky split is applied to the five baselines, the relative ranking may survive in the table, but the central claim that F-SE-LSTM 'outperforms existing state-of-the-art deep learning anomaly detection methods in terms of anomaly detection capability' is not established: the absolute metrics are inflated and the comparison is not a valid measure of generalization. A secondary form of test-set fitting appears in Section 4.4.3, where the sliding-window size T is chosen by comparing F1 on the test set. The architecture is plausible and the frequency-matrix idea may merit study, but the empirical evidence for the headline claim is broken.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes F-SE-LSTM, a supervised time-series anomaly-detection method that converts raw series into sliding-window samples of length 60 and then into a frequency matrix using a second sliding window of length T followed by FFT. SENet and LSTM are used to extract frequency-related features within and between time periods, and a DNN outputs binary anomaly labels. Experiments on Yahoo Webscope A1 and Numenta Anomaly Benchmark (AWS, Known, Traffic) report higher F1, recall, precision, and accuracy than C-LSTM, C-LSTM-AE, CNN-1D, TCN, and FFT-1D-CNN, together with competitive training time and parameter count. The central claim is that the frequency-matrix representation plus the SENet-LSTM architecture outperforms existing deep learning anomaly-detection methods.","tokens_in":15386,"tokens_out":6732,"duration_ms":70523,"significance":"The architecture is internally coherent, the parameter tables are detailed, and the code is made available, which are concrete strengths. The frequency-matrix construction is a plausible design idea, and the small parameter count (35,654) is attractive if the empirical claims hold. However, the empirical evaluation has a load-bearing flaw: overlapping sliding-window samples are randomly split into train/validation/test sets, so training and test windows share almost all raw observations and labels. The reported F1 scores therefore do not measure out-of-sample detection capability, and the headline outperformance claim is not supported by the evidence as presented. Because the contribution is empirical rather than derivational, the validity of the evaluation is decisive; the paper does not provide a valid test of its main claim.","major_comments":[{"comment":"The evaluation is not out-of-sample. In Eq. (1), M=L-N+1 and N=60, so consecutive rows of D overlap in 59 raw observations and every raw observation appears in up to 60 samples. Section 4.2 then states that random stratified sampling divides the dataset into 60/20/20 training/validation/test sets. Consequently, each test window has many training windows that differ by one or a few time steps and carry the same 'contains an outlier' label, so the classifier can memorize near-duplicate windows instead of detecting anomalies. The reported F1 values in Tables 6, 7, and 8 (e.g., 0.9799 on A1 and 0.9969 on Traffic) are therefore inflated, and the central claim that F-SE-LSTM outperforms state-of-the-art methods is not established. The same leaky split applies to the baselines, so the relative ordering in the tables is not by itself evidence against this problem.","section":"Section 4.2 and Eq. (1)"},{"comment":"The sliding-window size T appears to be selected on the test set, which is a second form of evaluation leakage. The section compares T=10,20,30,40,50 and concludes that T=30 is best because it yields the highest F1 scores in Table 5. The paper never states that Table 5 is computed on a validation set; given that the F1 results reported around it are test-set results, the choice of T is fitted to the test data. If T was actually chosen on validation, the text must say so explicitly; otherwise the final model's test numbers incorporate test-set information beyond the epoch-selection procedure described in Section 4.4.","section":"Section 4.4.3 and Table 5"},{"comment":"The reported improvements over the strongest baselines are very small in several cases (e.g., F-SE-LSTM 0.9799 versus CNN-1D 0.9765 on A1), yet all results come from a single run with random state fixed to 1 and no confidence intervals or significance tests are reported. The small margins, combined with the leakage in Major Comment 1, do not support the claim of consistent outperformance of the state of the art.","section":"Section 4.4.5 and Table 7"}],"minor_comments":[{"comment":"The text refers to 'logical regression' where 'logistic regression' is meant.","section":"Section 4.4.1"},{"comment":"The layer name 'A vgPood2d' is a typo for 'AvgPool2d', and the row 'Hadamard product the outputs of 1 and 2' is not a complete sentence.","section":"Table 1"},{"comment":"The symbol T is used both for the sliding-window size in the frequency matrix and for the DFT length in Eq. (5), which is confusing; consider using separate symbols.","section":"Section 3.1 and Eq. (5)"},{"comment":"The figures use curves and bars without numeric axis labels or value markers, so the reader cannot verify the claimed differences from the figures alone.","section":"Figures 5, 6, and 7"},{"comment":"The sentence 'Comparative results with other methods that the frequency matrix constructed by our method exhibits superior discriminative ability' is grammatically incomplete and should be rewritten.","section":"Conclusion"},{"comment":"There are numerous typographical errors such as 'di fferent', 'tra ffic', and 'e fficiency'; the manuscript should be proofread.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The rejection is based on the invalid evaluation, not on the potential of the underlying idea. The leakage from randomly splitting overlapping sliding-window samples invalidates every reported F1 score, and correcting it requires re-running all experiments with a non-overlapping temporal split, which is a substantial change rather than a local fix. The paper is empirical, so a valid experimental protocol is essential."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe paper is a coherent engineering combination: two sliding windows produce a frequency matrix, SENet handles channel-wise frequency dependencies, LSTM models the period dimension. The representation is new in the sense that the exact stack isn't in the cited prior work, and the parameter tables and GitHub code show care. That part is worth some credit.\n\nThe empirical support, though, doesn't hold up. Eq. (1) builds overlapping windows with N=60 and M=L−N+1, so each raw point appears in up to 60 samples. Section 4.2 then randomly splits those samples 60/20/20. A test window shares up to 59 of 60 points with training windows, and labels are assigned at the sample level (\"contains an outlier\"). That's information leakage, not independent evaluation. The reported F1 values are inflated, so the abstract's superiority claim is not established. The same leaky split applies to the baselines, so relative rankings might survive, but absolute metrics are meaningless for generalization.\n\nThere's also a secondary issue: Section 4.4.3 picks the sub-window size T by comparing test F1 across values, which is test-set fitting. No error bars or significance tests are reported, and the compared baselines are a narrow, not-current set.\n\nThe central flaw is load-bearing. A proper evaluation would split by time or use non-overlapping windows, tune T on validation, and report variance. I'd reject in current form. That said, the frequency-matrix idea and the architecture are plausible enough that a revised version with a valid evaluation could be worth another look. A serious referee could guide that revision, but the paper as written should not be accepted.\n\nRegards.","headline":"A coherent frequency-matrix architecture with a leaky evaluation: the central outperformance claim is unsupported.","tokens_in":15976,"tokens_out":2874,"would_cite":false,"duration_ms":29660,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Time-series anomalies too subtle for the time domain become separable when the series is recast as a frequency matrix; F-SE-LSTM uses that representation and reports consistent F1 gains over five deep baselines on Yahoo and NAB data.","keywords":["time series anomaly detection","fast Fourier transform","frequency matrix","Squeeze-and-Excitation networks","LSTM","sliding window","Yahoo A1","Numenta Anomaly Benchmark"],"falsifier":"Train F-SE-LSTM and its strongest baseline on samples taken only from the first portion of each sequence and test on samples from the remaining, non-overlapping portion, or hold out a set of complete time series. If the F1 advantage of the frequency matrix largely disappears under this temporally separated split, the claimed outperformance is an artifact of overlapping train and test windows; if the advantage persists, the representation itself is doing the work.","tokens_in":14900,"feed_emoji":"📈","tokens_out":8800,"duration_ms":84270,"temperature":0.7,"pith_summary":"The paper tries to establish that time-series anomaly detection improves when the input keeps frequency information in time order rather than analyzing raw amplitudes alone. It constructs a frequency matrix by sliding a window twice across a series and applying the fast Fourier transform to each row, then trains a network in which SENet weights frequency dependencies within a period and LSTM captures dependencies across periods. On the Yahoo A1 and Numenta Anomaly Benchmark datasets, the method reports higher F1, recall, precision, and accuracy than five existing deep anomaly detectors, with about 35,700 parameters and a 379-second training time on the reported hardware. A sympathetic reading of the contribution is that hidden anomalies—sequences that look similar in the time domain but differ at specific frequencies—are exactly what this representation is designed to expose.","feed_headline":"Frequency-matrix encoding lifts anomaly detection F1","feed_subtitle":"FFT plus channel attention and LSTM finds hidden anomalies that look alike in the time domain—with only 35k parameters.","key_machinery":"The central object is the frequency matrix $\\mathbf{S}_f \\in \\mathbb{R}^{H \\times (F+1)}$: sliding the sample window once to get $H=N-T+1$ overlapping rows of length $T$, then applying the fast Fourier transform row-wise and keeping the magnitude spectrum $y(n)=|X(n)|$ for $n=0,\\ldots,\\lfloor T/2\\rfloor$. Because the spectrum of a real sequence is conjugate-symmetric, only $\\lfloor T/2\\rfloor+1$ frequency bins carry information, so the matrix keeps compact frequency content while preserving which period each frequency profile came from. The model then delegates two complementary tasks to this matrix: SENet, a channel-attention network, squeezes each channel to a scalar and excites important frequency-channel combinations, extracting dependencies among frequencies inside the same period; LSTM processes the $H$ period rows as a sequence of hidden states, extracting dependencies among frequencies across periods; and a DNN reduces the LSTM output to a binary anomaly decision.","core_discovery":"The paper's central claim is that a frequency matrix built from two overlapping sliding windows is a better input representation for time-series anomaly detection than raw time-domain vectors, isolated frequency vectors, or time-domain matrices, and that the combination of SENet and LSTM is the right way to consume it. A length-60 sample is first cut into 31 overlapping length-30 subsequences; each subsequence is transformed by FFT into 16 amplitude bins ($F=\\lfloor 30/2\\rfloor=15$, plus the zero-frequency bin), producing a $31\\times 16$ frequency matrix $\\mathbf{S}_f$. SENet applies channel attention to emphasize useful frequency-channel combinations inside each period, LSTM reads the period rows as 31 time steps to model between-period frequency dependencies, and a DNN with dropout outputs the binary anomaly verdict. In the authors' experiments, F-SE-LSTM reaches F1 scores of 0.9799, 0.9728, 0.9256, and 0.9969 on the A1, AWS, Known, and Traffic datasets, an average F1 of 0.9688 against a best baseline average of 0.9566, and it does so with 35,654 parameters and a 379-second training time.","pith_inferences":["A testable extension of the paper's comparison would evaluate on temporally separated test windows or held-out complete series; the random split of overlapping windows is the one part of the protocol that a skeptical reader would want to see changed before trusting the margin.","The two-window FFT encoding is a general input recipe: any sequence model that can ingest a matrix, including attention-based models, could be swapped in for SENet and LSTM, and the encoding's value can be tested independently of the classifier.","The same encoding should transfer to multivariate time series by stacking per-channel frequency matrices, or to unsupervised reconstruction-based detection by training an autoencoder on the frequency matrix; neither case is explored in the paper.","The choice $T=30$ is a tunable balance between frequency resolution and the number of periods, so on longer series with stronger periodicity the optimum could shift; that is a directly testable hyperparameter question."],"forward_implications":["If the reported results hold, the frequency matrix encoding alone explains a large part of the gain: in the paper's tables, frequency-domain features beat time-domain features for kNN, logistic regression, SVM, decision trees, and random forests on all four datasets.","SENet plus LSTM is presented as a more stable architecture than CNN plus LSTM for this input, because it reaches the highest F1 on all four datasets whereas the CNN variant drops below LSTM-only on two of them.","The low parameter count of 35,654 and the 379-second training time mean the method is light enough to retrain frequently in network and IoT monitoring settings.","The method's motivating scenario becomes operational: anomalies that look like normal traffic in the time domain but differ at particular frequencies should be detectable, which is the case the paper demonstrates on the four benchmark datasets."],"supporting_citations":[{"why":"Supplies the FFT as the fast transform used to convert each row of the time matrix into a magnitude spectrum.","marker":"[18]"},{"why":"Supplies the Squeeze-and-Excitation channel-attention block used to weight frequency channels within a period.","marker":"[19]"},{"why":"Supplies the LSTM cell equations used to model dependencies across time periods.","marker":"[20]"},{"why":"Supplies the Yahoo A1 dataset, one of the two benchmark sources for all experiments.","marker":"[40]"},{"why":"Supplies the Numenta Anomaly Benchmark datasets AWS, Known, and Traffic.","marker":"[41]"},{"why":"Provides the C-LSTM deep baseline whose F1, recall, precision, and training cost the method must beat.","marker":"[36]"},{"why":"Provides the C-LSTM-AE baseline that combines a convolutional recurrent autoencoder for comparison.","marker":"[38]"},{"why":"Provides the CNN-1D baseline, the strongest comparator on average F1.","marker":"[26]"},{"why":"Provides the TCN baseline, the temporal-convolution method compared in the main results.","marker":"[27]"},{"why":"Provides the FFT-1D-CNN baseline, the nearest frequency-domain competitor that uses FFT features directly.","marker":"[16]"}],"fun_headline_variants":["Frequency matrix catches anomalies time domain misses","FFT-based matrix outperforms in anomaly detection F1","SENet+LSTM on frequency data finds hidden anomalies","Frequency representation sharpens anomaly detection","F-SE-LSTM: seeing anomalies in frequency space"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's load-bearing premise, stated in its data-preprocessing step, is that randomly partitioning the overlapping sliding-window samples into 60/20/20 train/validation/test sets yields independent evaluation data.","fun_headline_variants_meta":{"raw":{"variants":["Frequency matrix catches anomalies time domain misses","FFT-based matrix outperforms in anomaly detection F1","SENet+LSTM on frequency data finds hidden anomalies","Frequency representation sharpens anomaly detection","F-SE-LSTM: seeing anomalies in frequency space"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000204,"raw_usage":{"total_tokens":1414,"prompt_tokens":994,"completion_tokens":420,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":348}},"tokens_in":610,"tokens_out":420,"duration_ms":4634,"temperature":1.0,"reasoning_tokens":348,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:24:23.399857+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train F-SE-LSTM and its strongest baseline on samples taken only from the first portion of each sequence and test on samples from the remaining, non-overlapping portion, or hold out a set of complete time series. If the F1 advantage of the frequency matrix largely disappears under this temporally separated split, the claimed outperformance is an artifact of overlapping train and test windows; if the advantage persists, the representation itself is doing the work.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the FFT as the fast transform used to convert each row of the time matrix into a magnitude spectrum."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Squeeze-and-Excitation channel-attention block used to weight frequency channels within a period."},{"cited_title":"Laptev, S","cited_arxiv_id":null,"evidence_quote":"Supplies the Yahoo A1 dataset, one of the two benchmark sources for all experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Numenta Anomaly Benchmark datasets AWS, Known, and Traffic."},{"cited_title":"Kim, S.-B","cited_arxiv_id":null,"evidence_quote":"Provides the C-LSTM deep baseline whose F1, recall, precision, and training cost the method must beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the C-LSTM-AE baseline that combines a convolutional recurrent autoencoder for comparison."},{"cited_title":"Ullah, Q","cited_arxiv_id":null,"evidence_quote":"Provides the CNN-1D baseline, the strongest comparator on average F1."},{"cited_title":"A Temporal Anomaly Detection System for Vehicles utilizing Functional Working Groups and Sensor Channels","cited_arxiv_id":"2209.06828","evidence_quote":"Provides the TCN baseline, the temporal-convolution method compared in the main results."},{"cited_title":"Rahimi, A","cited_arxiv_id":null,"evidence_quote":"Provides the FFT-1D-CNN baseline, the nearest frequency-domain competitor that uses FFT features directly."}],"review_version":1}