{"id":"72e29bb1-1b01-4b51-8413-b72635fec35d","arxiv_id":"2501.00107","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A DQN agent that picks among six anomaly detectors using time series forest rewards achieves high F1 on two electricity datasets, but the evaluation is in-sample.","lead":"This paper trains a reinforcement learning agent to choose, at each time step, which of six anomaly detectors to trust for electricity consumption data, using time series forests to guess which detectors are correct when labels are missing. On the datasets tested, the selector beats most individual detectors on F1 score, but the agent is trained and evaluated on the same data, so the advantage may not hold on new data.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed edge over unsupervised detectors is computed on the same anomalous sequence used to train the TSF classifiers and the DQN policy, so the reported F1 measures label leakage and in-sample optimization rather than model-selection quality.","rationale":"The reader's weakest assumption is exactly the right one: the paper's headline result is an in-sample number. I checked the methodology sections and found no separate validation or test split for the RL policy or the TSF classifiers; the same anomalous dataset is used for TSF training (20% labels), DQN reward computation, and final evaluation. This is not a minor statistical caveat. Because the TSF classifiers are themselves label-trained on part of the evaluation set and are highly accurate on it, the DQN's reward signal approximates full access to the test labels. The comparison to unsupervised detectors that see no labels is therefore not a test of the proposed model-selection idea. A proper held-out evaluation could still support the claim, but the paper as written does not provide one, and the fixed seed / single-run results add further uncertainty. I agree fully with the reader's assessment and would keep the rejection. No additional concern seems more load-bearing than this one.","tokens_in":19496,"tokens_out":6704,"duration_ms":68634,"concrete_test":"Split the real anomalous dataset temporally into a training prefix and a held-out suffix, leaving a gap of at least one sliding-window length (6 steps) to prevent window overlap. Train the six TSFs and the DQN using labels only on the prefix, set each AD model's threshold from the prefix rather than from the known 5% anomaly ratio of the full set, and compute precision/recall/F1 on the suffix alone. Also compute KNN's metrics on the same suffix. If the proposed selector's held-out F1 no longer exceeds KNN's, the in-sample evaluation is the source of the claimed improvement.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that the proposed selector outperforms all candidate AD models—requires that the reported F1 estimates performance on data not used to fit the selector. The paper never provides such a split. Section 3.4 trains six TSF classifiers to predict whether each AD model's label is correct, using ground truth on 20% of Danomaly. Section 3.5 places the entire Danomaly in the DQN state, defines the reward as TSF predictions on 80% of Danomaly and direct ground truth on the remaining 20%, and explicitly reuses the same 20% labels used for TSF training. Section 4.4 trains the DQN for 3,000,000 time steps on the real anomalous dataset, and Section 5.3 reports precision/F1 on that same dataset. The TSF classifiers attain F1 0.91–0.98 on this same data, so the reward signal is nearly equivalent to full ground truth. The six unsupervised baselines receive no labels. The comparison therefore conflates the selector's contribution with the contribution of test-set labels and in-sample training; generalization to new time series is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RLAD, a model-selection framework for time-series anomaly detection (AD) in electricity-consumption data. Six unsupervised AD models (KNN, COPOD, ECOD, OSVM, IForest, USAD) are trained on normal data, and six Time Series Forest (TSF) classifiers are trained on 20% of the anomalous test sequence to predict whether each AD model's prediction is correct. A DQN agent then chooses, at each time step, which AD model's label to output; its reward is computed from TSF predictions on 80% of the anomalous sequence and ground-truth labels on the same 20% used for TSF training. The paper reports F1 scores on a real LEAD1.0 subset and a synthetic dataset, claiming the proposed selector outperforms all candidate AD models on the real data and all but KNN on the synthetic data, and also outperforms a GPT-4-based detector on the synthetic data. Additional experiments study reward-function variations, exploration epsilons, and anomaly types (global, local, clustered).","tokens_in":19806,"tokens_out":4978,"duration_ms":51882,"significance":"If the reported results were valid, the paper would offer a useful recipe for dynamic AD model selection with limited labels, and its comparison against an LLM-based detector is topical. The paper also contains a reasonably broad survey of unsupervised AD categories and a systematic ablation of reward designs. However, the paper's central quantitative claim rests on a flawed evaluation protocol: the DQN policy is trained and scored on the same anomalous sequence, with TSF proxies that are themselves trained on ground-truth labels from that sequence. Because the TSFs attain F1 scores of 0.91-0.98 on the real data, the reward signal is nearly equivalent to giving the agent the test-set labels during training, while the six baselines receive no labels at all. The reported F1 advantage is therefore evidence of label leakage and in-sample optimization rather than of model-selection quality. No code or data is provided to reproduce the experiments, and the claimed generalization to unseen time series is unsupported.","major_comments":[{"comment":"The evaluation trains and tests on the same anomalous dataset, so the central comparison is invalid. Section 3.4 trains six TSF classifiers on 20% of Danomaly using ground-truth labels. Section 3.5 defines the DQN reward as TSF predictions on 80% of Danomaly and direct ground truth on the same 20% used for TSF training, explicitly reusing those labels. Section 4.4 trains the DQN for 3,000,000 time steps on the real anomalous dataset, and Table 2 reports precision/F1 on that same dataset. Since the TSFs achieve F1 0.913-0.979 on this data (Table 4), the agent effectively receives near-perfect label information across the entire training sequence; the reported F1 of 0.727 on the real data reflects in-sample optimization, not model-selection quality. The six unsupervised baselines receive no labels, so the comparison conflates the selector's contribution with test-set labels. A held-out test sequence, with TSF training and DQN rewards confined to a disjunct training portion, is required to support any generalization claim.","section":"Secs. 3.4, 3.5, 4.4, 5.3"},{"comment":"The claimed superiority over unsupervised candidates is not robust even within the leaked evaluation. Table 7 shows that RLAD_Gtruth, which uses the full ground truth in its reward, achieves F1=0.793 on the real dataset while the proposed framework achieves F1=0.727. This indicates that the label-derived reward component is doing much of the work, and the proposed method does not even beat a fully label-rewarded variant. The paper should additionally compare against a semi-supervised baseline trained directly on the same 20% labels and evaluated on the remaining 80% (with the DQN likewise trained only on a training split), to demonstrate that the selector adds value beyond simply using the labels.","section":"Secs. 5.3 and 5.5, Table 7"},{"comment":"The anomaly threshold for every candidate AD model is set to the 5% rank criterion, using the known anomaly percentage of each dataset. This transfers label-derived information (the global anomaly rate) into every baseline's decision rule, weakening the claim that the six candidate models are unsupervised. The paper reports no sensitivity analysis for this criterion, so it is unclear how much of the baseline performance, and of the proposed method's advantage, depends on this choice. The threshold should be treated as a hyperparameter and varied, or selected using only the normal training data.","section":"Sec. 4.3"}],"minor_comments":[{"comment":"The reward specification text says 'rFN is set to -1.5, and rFN is set to -3'; the first value should be rFP = -1.5 and the second rFN = -3, as reflected in Equation (6).","section":"Sec. 3.5"},{"comment":"The text refers to 'the same decaying exploration epsilon of Figure 2', but Figure 2 is the LLM prompt template; the decaying-epsilon schedule is presumably the one described in Section 4.4. The figure cross-references should be corrected.","section":"Sec. 5.7.1"},{"comment":"The text says 'as shown in Tables 14' but the referenced table is Table 14; also, the table caption 'F1score of AD Model' is ungrammatical and the table header 'Global Local Clustered' should be marked as F1 scores.","section":"Sec. 5.8"},{"comment":"The GPT-4 comparison is made only on the synthetic dataset, where KNN already achieves F1=1; this comparison is not informative about the proposed method's value on the real dataset, and no statistical significance or variance across prompts is reported.","section":"Sec. 5.4"},{"comment":"The abstract claims the approach works 'without explicitly depending on ground truth labels', but Section 3.4 requires 20% labels for TSF training and Section 3.5 uses labels in the reward. The wording should be revised to 'with a limited portion of ground truth labels'.","section":"Abstract and Sec. 3"}],"recommendation":"reject","confidential_remarks":"The manuscript's central empirical claim is not supported because the DQN policy and TSF classifiers are trained and evaluated on the same anomalous sequence, with the reward using ground-truth labels from that sequence. This is not a minor fix: the evaluation protocol must be redesigned with a proper held-out split, and the comparison against purely unsupervised baselines needs to account for the label budget. Given that prior work [1] already applies RL-based model selection to time-series AD, the incremental contribution is modest and would require substantially stronger evidence. I recommend rejection, though the authors could attempt a resubmission with a valid protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper has a legitimate new combination—replacing the ground-truth reward in [1] with TSF correctness classifiers, and adding KNN to the detector pool—and the reward-function/epsilon study is thorough. But the headline result is not supported. The DQN is trained and evaluated on the same Danomaly; 20% of that sequence's ground truth goes into the reward, the same 20% used to train the TSFs, and the whole sequence is part of the state. Section 4.4 trains for 3M steps on this data, then Section 5.3 reports precision/F1 on the same data. The six unsupervised baselines never see a label. So the comparison measures label leakage and in-sample optimization, not model-selection skill. This is not a minor caveat; the central claim depends on it.\n\nWhat is genuinely useful: the idea of using TSF predictions as a dense reward proxy is a sensible way to cut label cost, and if the evaluation were redone on a truly held-out time period, the setting—20% labels to pick among six detectors—would be practically interesting. The anomaly-type results (Table 14) are also informative, even if they are secondary. The baseline implementations, hyperparameter choices, and libraries are documented; the citation to [1] is proper, and the incremental framing is honest.\n\nSofter issues: the title and abstract say 'unsupervised,' but using 20% labels from the evaluation sequence is semi-supervised/transductive. The threshold criterion (top 5% scores) makes precision=recall for every baseline by construction, which should be stated rather than left to look coincidental. The GPT-4 comparison is on one small synthetic dataset and adds little.\n\nBottom line: this is a paper for a reading group on evaluation leakage and for anyone designing detector-selection methods. The recipe is reproducible and the idea is worth engaging, but the reported numbers should not be taken at face value. A serious referee should ask for a held-out temporal split, a clear statement of the semi-supervised setting, and a comparison where the baselines receive the same partial-label information if the claim is about label efficiency. With that revision, the paper could become a modest but solid incremental contribution. I would not cite the current results, but I would send the paper to peer review expecting major revision.","headline":"The TSF-reward twist and the anomaly-type comparison are worth a look, but the central accuracy claim rests on training and scoring the selector on the same anomalous sequence, so the reported F1 is not evidence of generalization.","tokens_in":20251,"tokens_out":4984,"would_cite":false,"duration_ms":73535,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An RL-trained model picker, choosing the right detector at each time step, can beat every individual anomaly detector on real electricity-consumption data.","keywords":["anomaly detection","time series","reinforcement learning","model selection","time series forest","DQN","electricity consumption","unsupervised learning"],"falsifier":"Run the trained policy on a held-out anomalous sequence from meters not used in training, rewarding it only through the TSF classifiers, and compare its F1 with KNN's; a result at or below KNN's 0.702 on the real dataset would contradict the claim that the selector outperforms all candidate detectors.","tokens_in":19327,"feed_emoji":"⚡","tokens_out":5657,"duration_ms":54718,"temperature":0.7,"pith_summary":"This paper tries to show that an agent trained with reinforcement learning can decide, at every time step, which of six unsupervised anomaly detectors to trust for electricity-consumption data, using only a small fraction of ground-truth labels. The proposed selector dynamically picks among KNN, COPOD, ECOD, one-class SVM, isolation forest, and USAD, guided by six time-series-forest classifiers that estimate whether each detector's prediction is correct. On the real smart-meter dataset the paper reports a precision of 0.977 and an F1 score of 0.727, beating every individual detector in precision and F1; on a synthetic dataset it reaches an F1 of 0.989, second only to KNN's perfect 1.0. If this holds, operators can obtain adaptive anomaly detection without large labeled sets, and can keep high performance across global, local, and clustered anomaly types.","feed_headline":"RL-trained model picker beats every single detector on real power data","feed_subtitle":"It switches among six detectors each step and lifts real-meter F1 to 0.727 with partial labels.","key_machinery":"The load-bearing object is the DQN policy combined with six TSF classifiers, one per candidate detector. Each TSF is trained on 20% of the anomalous sequence to classify whether its detector's predicted label is correct, using input features that include the window, the rescaled anomaly score, the empirical threshold, the predicted label, and two confidence scores: distance-to-threshold and prediction consensus. The DQN uses these TSF outputs as pseudo-labels for most reward computation, so the agent learns to select the detector most likely to be right without needing full ground truth. The asymmetric reward function rewards true positives more than true negatives and penalizes false negatives more than false positives.","core_discovery":"The central claim is that model selection for unsupervised anomaly detection can be learned as a sequential decision problem: a deep Q-network observes the current sliding window, all six detectors' anomaly scores, their empirical thresholds, predicted labels, and two confidence scores, then picks one detector; the reward compares the chosen detector's label with a TSF classifier's estimate of correctness on 80% of points and with ground truth on the remaining 20%. The authors report that this selector outscores all six candidate detectors on the real dataset and all but KNN on the synthetic dataset, and that it stays accurate across datasets separately injected with global, local, and clustered anomalies. The intended message is that a detector pool plus a modest label budget can replace the search for a single universally best algorithm.","pith_inferences":["Because the paper's headline numbers are computed on the same anomalous sequence used to train the DQN, the practical advantage would need confirmation on a held-out sequence; the 20% label subset is also reused for TSF training and reward, so the effective label cost may be higher than stated.","The selector may be learning a meta-signal about detector reliability rather than the anomaly structure itself; if so, retraining on a new dataset's detector outputs could work even when the raw electricity patterns differ.","The TSF pseudo-labelers could be replaced by any calibrated classifier, making the scheme a general 'detector of detectors' approach applicable beyond electricity data.","A direct testable extension is to vary the 20% label fraction downward and measure the precision/F1 trade-off, which would map how much label savings the method truly buys."],"forward_implications":["A pool of imperfect detectors plus a learned switcher can outperform the best single detector in the pool on real electricity data.","Only about 20-30% of anomalous instances need labels to train the selector, relaxing the label bottleneck in unsupervised anomaly detection deployments.","Detector rankings change with anomaly type (global, local, clustered), so a fixed best model is not reliable across settings.","Reward design matters: the original asymmetric reward with decaying exploration beats both constant and adaptive reward alternatives.","On the synthetic benchmark, the RL selector exceeds a chain-of-thought prompted LLM anomaly detector (F1 0.989 vs 0.913)."],"supporting_citations":[{"why":"Supplies the initial RL-based model selection idea for anomaly detection and the two confidence scores reused here.","marker":"[1]"},{"why":"Provides the time series forest classifier method used to predict whether each detector's label is correct.","marker":"[23]"},{"why":"Supplies the real smart-meter dataset (LEAD1.0) from which the anomalous evaluation split is taken.","marker":"[62]"},{"why":"Provides the DQN implementation used to train the selection policy.","marker":"[68]"},{"why":"Provides off-the-shelf implementations of several candidate detectors in the pool.","marker":"[65]"},{"why":"Defines the KNN anomaly detector, the strongest single baseline in both datasets.","marker":"[32]"},{"why":"Defines the Isolation Forest anomaly detector, one of the pooled candidates.","marker":"[46]"},{"why":"Defines the USAD anomaly detector, the deep-learning candidate in the pool.","marker":"[53]"}],"fun_headline_variants":["RL picks best anomaly detector per step, beats all on real data","RL adaptively selects detector, outperforming any fixed choice on real data","Adaptive detector switcher outperforms all fixed methods on real meters","ML model picking beats every single anomaly detector on power data","RL-driven detector selection beats all six algorithms on real data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported outperformance assumes that the F1 scores, measured on the same anomalous dataset the DQN was trained on, carry over to new time series; the policy's rewards come from TSF predictions on 80% of that dataset and ground truth on the same 20% used to train the TSFs.","fun_headline_variants_meta":{"raw":{"variants":["RL picks best anomaly detector per step, beats all on real data","RL adaptively selects detector, outperforming any fixed choice on real data","Adaptive detector switcher outperforms all fixed methods on real meters","ML model picking beats every single anomaly detector on power data","RL-driven detector selection beats all six algorithms on real data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000591,"raw_usage":{"total_tokens":2786,"prompt_tokens":977,"completion_tokens":1809,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":1721}},"tokens_in":593,"tokens_out":1809,"duration_ms":13390,"temperature":1.0,"reasoning_tokens":1721,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:59:11.434307+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained policy on a held-out anomalous sequence from meters not used in training, rewarding it only through the TSF classifiers, and compare its F1 with KNN's; a result at or below KNN's 0.702 on the real dataset would contradict the claim that the selector outperforms all candidate detectors.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the initial RL-based model selection idea for anomaly detection and the two confidence scores reused here."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the time series forest classifier method used to predict whether each detector's label is correct."},{"cited_title":"Gulati and P","cited_arxiv_id":null,"evidence_quote":"Supplies the real smart-meter dataset (LEAD1.0) from which the anomalous evaluation split is taken."},{"cited_title":"Raffin, A","cited_arxiv_id":null,"evidence_quote":"Provides the DQN implementation used to train the selection policy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides off-the-shelf implementations of several candidate detectors in the pool."},{"cited_title":"Angiulli and C","cited_arxiv_id":null,"evidence_quote":"Defines the KNN anomaly detector, the strongest single baseline in both datasets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Isolation Forest anomaly detector, one of the pooled candidates."},{"cited_title":"Audibert, P","cited_arxiv_id":null,"evidence_quote":"Defines the USAD anomaly detector, the deep-learning candidate in the pool."}],"review_version":1}