{"id":"88a983b0-2c72-42c5-aaca-75ed880778dc","arxiv_id":"2502.03245","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A wavelet-augmented autoencoder with a reinforcement-learning boundary tuner is proposed for multivariate time-series anomaly detection, but the reported performance figures rest on circular labels.","lead":"The paper combines wavelets, an autoencoder, and a Q-learning agent to detect anomalies in multivariate sensor data, then evaluates the method on aero-engine data. The core experimental claim is not supported because the anomaly labels used for scoring are built from the same reconstruction errors the model uses to detect anomalies.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section IV's evaluation is circular: the test labels are defined by thresholding the same reconstruction-error and uncertainty scores the model uses to detect anomalies, so Table II measures self-consistency, not real anomaly-detection performance on C-MAPSS.","rationale":"The reader's weakest-assumption analysis identifies exactly the same load-bearing flaw: the evaluation labels are derived from the model's own reconstruction-error and uncertainty scores. This is not a minor methodological quibble; it invalidates the only quantitative evidence for the paper's central claim that the proposed method \"consistently outperformed the LSTM-AE model in all metrics.\" The paper explicitly concedes that true labels are unavailable and then substitutes a labeling rule that thresholds the same features used for detection. Because the model is trained and calibrated to separate points by reconstruction error and uncertainty, high agreement with such labels is guaranteed by construction, and the LSTM-AE comparison is biased: LSTM-AE does not use the uncertainty component, yet it is scored against labels that partly depend on that component. The wavelet preprocessing and the idea of calibrating an autoencoder boundary with synthetic anomalies are reasonable ingredients, and they could in principle be validated, but this paper provides no valid external validation. No code, formal verification, or independent benchmark supports the reported numbers. The same circularity would also affect any ablation or error-bar analysis, because the target variable itself is model-dependent. Therefore the preprint should not be accepted as a demonstration of the proposed method. The appropriate disposition remains reject, matching the reader's verdict; I see no reason to adjust it.","tokens_in":7645,"tokens_out":3061,"duration_ms":28954,"concrete_test":"Re-run the experiment on C-MAPSS FD001 using externally derived ground-truth labels: declare a test window anomalous when the engine's remaining useful life falls below a fixed threshold (e.g., 30 cycles) or when a known fault-insertion point is crossed, and never use reconstruction-error or uncertainty thresholds to form the test labels. Compute precision, recall, accuracy, and F1 for the proposed method and LSTM-AE against these external labels. If the proposed method no longer beats LSTM-AE by the reported margins, the circular-labeling concern is confirmed and the central claim collapses.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim rests on a circular evaluation. In Section IV, the paragraph beginning \"We employ several evaluation metrics\" states: \"instances with synthetic anomalies, high reconstruction errors, and high uncertainty in the latent space are treated as anomalies... Conversely, data with low reconstruction error and low uncertainty are considered normal.\" The proposed detection rule in Algorithm 1 (lines 11-13) labels a point anomalous exactly when reconstruction error en exceeds the RL-adjusted threshold θn, and the high-uncertainty category is computed from Monte Carlo dropout in the same latent space. Thus the \"ground truth\" for precision, recall, accuracy, and F1 in Table II is constructed from the model's own anomaly scores. A threshold on en will score well against a label set that is itself a threshold on en, so the reported outperformance over LSTM-AE does not demonstrate detection of real anomalies. C-MAPSS has run-to-failure trajectories and remaining-useful-life annotations that could supply external anomaly labels, but the paper does not use them; instead, synthetic anomalies are also counted as test anomalies even though they were generated and used during training.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an unsupervised anomaly detection framework for multivariate time-series data that combines discrete wavelet transform (DWT) feature extraction, a convolutional neural network autoencoder (CNN-AE), and a Q-learning-based agent intended to calibrate the decision boundary using synthetic anomalies. The method is evaluated on the C-MAPSS aero-engine dataset and compared against an LSTM-AE baseline. The central empirical claim, stated in Section IV, is that the proposed method outperforms LSTM-AE on precision, recall, accuracy, and F1-score. However, the evaluation labels are constructed from the model's own reconstruction-error and uncertainty scores, making the reported metrics circular; additionally, the RL formalism is internally inconsistent between the equations and the algorithm. The paper concludes by suggesting future work with DQN or policy gradient methods.","tokens_in":7865,"tokens_out":4296,"duration_ms":39271,"significance":"If the empirical claim were valid, the integration of wavelet-based features, uncertainty estimation via Monte Carlo dropout, and RL-driven decision-boundary calibration could offer a meaningful contribution to unsupervised anomaly detection in multivariate time-series. The paper is clearly written in terms of structure, and it addresses a real challenge: the scarcity of anomalous labels. Credit is due for attempting to incorporate uncertainty and synthetic anomalies into boundary calibration on a standard benchmark dataset (C-MAPSS). However, the central evaluation is circular, and the RL component as described is not a well-defined algorithm. Because the claimed superiority over LSTM-AE rests on these defective pillars, the paper does not currently establish its stated contribution.","major_comments":[{"comment":"The evaluation is circular: the ground-truth labels used to compute precision, recall, accuracy, and F1 are defined by thresholding the same scores the model uses for detection. The paragraph states: 'instances with synthetic anomalies, high reconstruction errors, and high uncertainty in the latent space are treated as anomalies... Conversely, data with low reconstruction error and low uncertainty are considered normal.' Algorithm 1 (lines 11-13) predicts a label based on whether the reconstruction error e_n exceeds the adjusted threshold θ_n, and the uncertainty grouping is also derived from the model's latent-space representations. Thus, the reported metrics in Table II measure agreement between two thresholds on the same model outputs, not detection of actual anomalies. The comparison with LSTM-AE does not resolve this, because the labels are constructed from the proposed model's scores, not from independent ground truth. Furthermore, synthetic anomalies are included in the test labels even though they were generated and used during training, which constitutes data leakage. C-MAPSS provides run-to-failure trajectories from which anomaly labels could be derived (e.g., segments near the end of life), but the paper does not use them; the authors also explicitly acknowledge 'true labels for the anomalies are not available.' This invalidates the central empirical claim that the proposed method outperforms LSTM-AE.","section":"Section IV, 'We employ several evaluation metrics' paragraph; Table II"},{"comment":"The RL formalism is internally inconsistent. Eq. (5) defines the action as a tuple of continuous adjustments, a_n = (Δθ_enc, Δθ_n), and Eq. (6) defines the next state as s_{n+1} = f_enc(w_{n+1}; θ_enc + Δθ_enc). However, Algorithm 1 line 10 applies only θ_n ← θ_n + Δθ_n, and line 15 sets s_{n+1} = s_n, ignoring the encoder update and the next-state equation. Line 19 updates the encoder parameters by minimizing L_total, not by the RL action. Moreover, Section IV states 'two possible actions for the RL agent, classify data as normal or classify it as abnormal,' which is a discrete classification action, not the continuous boundary/encoder adjustment of Eq. (5). Tabular Q-learning is then applied to a state space consisting of continuous latent representations, for which tabular Q-learning is not well defined. As written, the RL component is not a concrete, reproducible algorithm, and the claimed mechanism of dynamic decision-boundary calibration via RL is not actually implemented.","section":"Section III-B, Eqs. (5)-(6), Eq. (10), Algorithm 1"},{"comment":"The reward function in Eq. (8) uses labels for synthetic anomalies and assumes all other data are normal (y_n = 0). This is a supervised signal, contradicting the paper's 'unsupervised anomaly detection' framing. More importantly, the synthetic anomalies are generated by perturbing normal signals (cyclic, sudden drift, gradual drift), and these same anomalies are used both to train the RL agent and to define part of the test labels. The paper provides no evidence that these synthetic perturbations are representative of real C-MAPSS degradation anomalies; indeed, Section IV states 'the exact point of abnormality is uncertain.' The calibrated decision boundary may therefore be tuned to artifacts of the perturbation method rather than to actual faults, and the reported performance on synthetic anomalies does not transfer to real anomalies. This is a load-bearing assumption that is not validated.","section":"Section III-B, Eq. (8); Section IV, synthetic anomaly generation"},{"comment":"The experimental comparison lacks statistical rigor: no standard deviations, no multiple independent runs, no hyperparameter sensitivity analysis, and only a single baseline (LSTM-AE). The reported differences (e.g., F1 0.931 vs. 0.802) could be within random variation, especially given the small evaluation set and the circular labeling. Without error bars or a significance test, the claim of 'consistently outperformed' is not supported. Even if the circularity were resolved, the paper would need additional baselines and repeated trials to demonstrate a reliable advantage.","section":"Section IV, experimental setup and Table II"}],"minor_comments":[{"comment":"There is a typo in the affiliation: 'Concordia Univerity' should be 'Concordia University.'","section":"Title page"},{"comment":"Eq. (11) defines L_total = L_recon - η Q(s_n, a_n), but Algorithm 1 line 19 sums over all n, Σ_n Q(s_n, a_n). The notation should be harmonized to avoid ambiguity about whether the Q-term is a per-sample or batch term.","section":"Section III-B, Eq. (11) and Algorithm 1 line 19"},{"comment":"The figure caption text says 'yellow circles represent real abnormal data points that were classified as high uncertainty by the model.' Since no ground-truth anomaly labels are used, these points should be described as 'high-uncertainty data points,' not 'real abnormal data points,' to avoid implying an external ground truth.","section":"Section IV, Fig. 5 description"},{"comment":"Table I lists 'Number of episode' (should be 'episodes'); also, the text says 'We initialize a Q-table with zeros,' but the state and action spaces as described in Section III-B are continuous, so a Q-table would be infinite. This inconsistency should be clarified.","section":"Section IV, Table I and text"}],"recommendation":"reject","confidential_remarks":"The circular evaluation is severe and central: the labels are defined by thresholding the model's own reconstruction errors and uncertainty scores, so Table II cannot support the claimed superiority over LSTM-AE. The RL formulation is also not internally consistent, and the use of synthetic anomalies in both training and testing creates leakage. These are load-bearing issues that would require a substantially redesigned evaluation (e.g., using C-MAPSS run-to-failure trajectories to define anomaly labels, and a proper train/test split with multiple runs) to be fixable within the manuscript's scope. The paper is clearly written and addresses a relevant problem, but in its current form it does not establish its contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's novelty is combinatorial: wavelet coefficients fed to a CNN autoencoder, with a Q-learning agent adjusting the decision boundary and synthetic anomalies injected to guide calibration. That specific pipeline isn't in the cited literature, and the paper explains it clearly enough to reproduce the architecture. The latent-space figure is a nice touch, and the wavelet preprocessing is a reasonable idea.\n\nThe problem is the evaluation, and it's a load-bearing one. In Section IV, the authors state that true labels aren't available, so they \"devise a labeling strategy\": points with high reconstruction error and high uncertainty are called anomalies, and low error plus low uncertainty are called normal. But the detector's own rule is exactly that—Algorithm 1 labels a point anomalous when reconstruction error exceeds an adjusted threshold, and the uncertainty grouping comes from Monte Carlo dropout in the same latent space. So the precision, recall, accuracy, and F1 in Table II measure how well a threshold on the reconstruction error matches a label set that is itself a threshold on that same error. That's circular, and it means the reported outperformance over LSTM-AE says nothing about detecting real anomalies in C-MAPSS. The dataset has run-to-failure trajectories with remaining-useful-life annotations that could supply external labels; the paper doesn't use them. It also counts synthetic anomalies as test anomalies even though they were used during training, which inflates the numbers.\n\nThere are secondary issues. The RL formalism is inconsistent: Eq. (5) says the action updates both encoder parameters and the boundary, but Algorithm 1 only updates the boundary, and line 15 sets the next state to the current state. The reward is a classification reward on synthetic labels, so the \"reinforcement learning\" reduces to a threshold search. There are no error bars, no ablations, and the only baseline is a plain LSTM-AE, which is not a competitive modern comparison.\n\nI agree with the stress-test: this is not a manufactured flaw. The circular evaluation breaks the central claim. The ingredients—wavelet features and synthetic-anomaly calibration—are worth keeping, but they need to be validated against real anomaly labels, and the RL part needs to actually do what the equations say.\n\nI wouldn't send this to referees as is. A desk reject with an invitation to resubmit after a proper C-MAPSS evaluation (using RUL-based labels, no synthetic anomalies in the test set) and a cleaned-up RL formulation would be the right call. As presented, the paper doesn't justify referee time.","headline":"The pipeline combines known parts in a new way, but the evaluation is circular: the test labels are thresholded versions of the same scores the detector uses, so Table II doesn't demonstrate real anomaly detection.","tokens_in":8397,"tokens_out":2733,"would_cite":false,"duration_ms":25052,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A reinforcement-learning agent that calibrates the decision boundary inside an autoencoder's latent space is claimed to lift unsupervised anomaly-detection F1 to 0.931 on the C-MAPSS aero-engine dataset.","keywords":["unsupervised anomaly detection","multivariate time series","reinforcement learning","Q-learning","autoencoder","wavelet transform","decision boundary calibration","C-MAPSS"],"falsifier":"Re-run the comparison on C-MAPSS using the actual degradation labels (for instance, windows after the known failure-progression threshold) instead of pseudo-labels derived from reconstruction error and uncertainty; if the proposed method's precision and recall drop to the baseline's level, the reported superiority was an artifact of circular labeling.","tokens_in":7371,"feed_emoji":"⚙️","tokens_out":6602,"duration_ms":56585,"temperature":0.7,"pith_summary":"The paper tries to establish that unsupervised anomaly detection in multivariate time series can be improved by letting a reinforcement learning agent calibrate the detector's decision boundary inside an autoencoder's latent space. The authors argue that scarce anomaly labels inflate false negatives, and that synthetic anomalies plus RL exploration can compensate. On the C-MAPSS aero-engine dataset, the method is reported to reach precision 0.921, recall 0.942, accuracy 0.911, and F1 0.931, beating an LSTM-AE baseline on every metric. A sympathetic reader would care because the approach addresses a practical setting where true anomaly labels are unavailable.","feed_headline":"RL-calibrated autoencoder hits 0.931 F1 on aero-engine anomalies","feed_subtitle":"Wavelet features plus a Q-learning agent sharpen the normal-versus-anomaly boundary, beating LSTM-AE on every metric.","key_machinery":"The load-bearing mechanism is an RL loop around the latent space. The state is the latent code $z_n = f_{\\text{enc}}(w_n; \\theta_{\\text{enc}})$; the action is $a_n = (\\Delta\\theta_{\\text{enc}}, \\Delta\\theta_n)$; and the reward is $R(s_n, a_n) = R_{\\text{sep}} + R_{\\text{acc}}$, where $R_{\\text{sep}}$ is the squared distance between normal and synthetic-anomaly centroids and $R_{\\text{acc}}$ is +1 or -1 for correct or incorrect classification under the current threshold. Training minimizes $L_{\\text{total}} = L_{\\text{recon}} - \\eta Q(s_n, a_n)$, so reconstruction fidelity and expected future reward pull in the same direction. Synthetic anomalies injected into training provide the supervised reference points that let the agent calibrate the boundary.","core_discovery":"The central claim is that the combination of wavelet-based multiresolution features, a convolutional autoencoder, and an RL-calibrated boundary yields a better-performing unsupervised anomaly detector. The discrete wavelet transform turns each sliding window into an image-like coefficient stack; the CNN autoencoder learns a three-dimensional latent space; and a Q-learning agent adjusts both encoder parameters and the threshold so that normal data and synthetic anomalies form separated clusters. The paper reports that this pipeline outperforms LSTM-AE on all metrics, with F1 0.931 versus 0.802.","pith_inferences":["Editorial inference: the reported evaluation is circular in the sense the paper itself describes, because the pseudo-labels come from high reconstruction error and high uncertainty — exactly the anomaly scores the detector uses — so a fair comparison would need independent labels.","Editorial inference: the RL agent as described adjusts a threshold and encoder gradients, so the improvement over LSTM-AE could come chiefly from the wavelet inputs or the CNN architecture rather than from RL per se.","Editorial inference: a direct test would replace the Q-learning update with a simple percentile threshold on reconstruction error; if the RL gain disappears, boundary calibration is not the active ingredient.","Editorial inference: the framework suggests a broader recipe — synthetic anomaly injection plus an explicit separation reward could calibrate any reconstruction-based detector, not just this autoencoder."],"forward_implications":["If the reported numbers hold, the same three-stage recipe — wavelet features, CNN autoencoder, and RL threshold calibration — should transfer to other multivariate monitoring tasks where anomalies are rare.","A calibrated boundary implies fewer false negatives, which matters in safety-critical monitoring such as engine health.","Synthetic anomaly injection becomes a viable way to supervise the boundary without real labels, meaning practitioners can design anomaly types for their own failure modes.","The RL component gives the detector an adaptive threshold that can track changes in the latent-space distribution over time."],"supporting_citations":[{"why":"Supplies the C-MAPSS multivariate aero-engine dataset that the experiment and the reported metrics are based on.","marker":"[20]"},{"why":"Provides the discrete wavelet transform formulation used to turn each sliding window into multiresolution coefficients.","marker":"[18]"},{"why":"Defines LSTM-based autoencoder anomaly detection, the baseline family the proposed method is compared against.","marker":"[8]"},{"why":"Earlier LSTM anomaly-detection method that motivates the LSTM-AE comparison.","marker":"[9]"},{"why":"Introduces calibrated one-class classification for unsupervised time-series anomaly detection, the calibration idea being adapted here.","marker":"[17]"},{"why":"LSTM-autoencoder reconstruction-based anomaly detection that anchors the baseline approach.","marker":"[19]"},{"why":"Earlier aero-engine anomaly detection work that this pipeline extends.","marker":"[16]"}],"fun_headline_variants":["Wavelet-RL autoencoder beats LSTM-AE on multivariate anomaly detection","RL-calibrated wavelet autoencoder hits 0.931 F1 in time-series anomaly detection","Unsupervised anomaly detection improved via RL calibration and wavelet features","Q-learning tuning boosts autoencoder anomaly detection in multivariate time-series","RL and wavelet features sharpen anomaly boundary in unsupervised detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that points with high reconstruction error and high latent uncertainty really are the anomalies, so the ground truth used to compute precision, recall, and F1 is built from the same signal the detector uses to decide; if that labeling is wrong, the reported advantage over LSTM-AE is not grounded.","fun_headline_variants_meta":{"raw":{"variants":["Wavelet-RL autoencoder beats LSTM-AE on multivariate anomaly detection","RL-calibrated wavelet autoencoder hits 0.931 F1 in time-series anomaly detection","Unsupervised anomaly detection improved via RL calibration and wavelet features","Q-learning tuning boosts autoencoder anomaly detection in multivariate time-series","RL and wavelet features sharpen anomaly boundary in unsupervised detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000457,"raw_usage":{"total_tokens":2227,"prompt_tokens":814,"completion_tokens":1413,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":430,"completion_tokens_details":{"reasoning_tokens":1319}},"tokens_in":430,"tokens_out":1413,"duration_ms":10719,"temperature":1.0,"reasoning_tokens":1319,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T05:22:03.186274+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the comparison on C-MAPSS using the actual degradation labels (for instance, windows after the known failure-progression threshold) instead of pseudo-labels derived from reconstruction error and uncertainty; if the proposed method's precision and recall drop to the baseline's level, the reported superiority was an artifact of circular labeling.","supporting_citations":[{"cited_title":"User’s guide for the commercial modular aero-propulsion system simulation (c-mapss),","cited_arxiv_id":null,"evidence_quote":"Supplies the C-MAPSS multivariate aero-engine dataset that the experiment and the reported metrics are based on."},{"cited_title":"A hybrid model based on discrete wavelet transform (DWT) and bidirectional recurrent neural networks for wind speed prediction,","cited_arxiv_id":null,"evidence_quote":"Provides the discrete wavelet transform formulation used to turn each sliding window into multiresolution coefficients."},{"cited_title":"Unsupervised anomaly detection in time series using LSTM-based autoencoders,","cited_arxiv_id":null,"evidence_quote":"Defines LSTM-based autoencoder anomaly detection, the baseline family the proposed method is compared against."},{"cited_title":"Long short term memory networks for anomaly detection in time series","cited_arxiv_id":null,"evidence_quote":"Earlier LSTM anomaly-detection method that motivates the LSTM-AE comparison."},{"cited_title":"Calibrated one-class classification for unsupervised time series anomaly detection,","cited_arxiv_id":null,"evidence_quote":"Introduces calibrated one-class classification for unsupervised time-series anomaly detection, the calibration idea being adapted here."},{"cited_title":"LSTM-autoencoder-based anomaly detection for indoor air quality time-series data,","cited_arxiv_id":null,"evidence_quote":"LSTM-autoencoder reconstruction-based anomaly detection that anchors the baseline approach."},{"cited_title":"Aero-engines anomaly detection using an unsupervised Fisher autoencoder,","cited_arxiv_id":null,"evidence_quote":"Earlier aero-engine anomaly detection work that this pipeline extends."}],"review_version":1}