{"id":"b8537e93-f6f1-4c03-a6e0-e5f4feb7a238","arxiv_id":"2501.11959","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"NRdetector combines positive-unlabeled learning, confidence-based sample selection, and a temporal smoothness-and-separability loss to predict point-level anomalies from noisy segment labels, and reports the best F1 on five datasets.","lead":"NRdetector turns coarse segment-level labels, where many anomalous segments are unlabeled, into point-level anomaly predictions for multivariate time series. The authors report better F1 scores than 13 baselines on five public benchmarks, with code released.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never defines the T-dimensional point-level score h(X) used by Eq. (6) and Stage-2; Eq. (2) pools temporal embeddings into one segment vector, so the TC loss and point ranking may rest on an undefined quantity.","rationale":"The reader's weakest_assumption targets exactly the gap between the GAP-pooled segment representation in Eq. (2) and the T-dimensional h(X) invoked in Eq. (6). Reading the paper in good faith, the most natural repair is that h(X) is intended to be the sequence of pre-pooling temporal embeddings, and that each h(X_i)_j should have been a scalar score (e.g., sigma(w^T h_t)) rather than a d-vector; alternatively, the classifier may have an unshown T-dimensional output layer. Either repair is small and testable against the released code, so the concern does not justify rejection by itself. But the central claim is explicitly about point-level detection, and both the TC loss and Stage-2 ranking depend on the undefined per-point score; this is the most load-bearing spot. I agree with the reader's choice. The experiments also lack error bars and use per-dataset hyperparameters (Table 10), but the point-level scoring issue is more fundamental because if it fails, the method's point-level outputs have no specified computation. The verdict remains CONDITIONAL pending code inspection and a rewritten Section 4.2.3.","tokens_in":21510,"tokens_out":5082,"duration_ms":53612,"concrete_test":"Run the released NRdetector code on the EMG dataset with a debug hook that records the shapes and values of the tensors entering L_smooth (Eq. 6) and the Stage-2 point-ranking step. Verify whether h(X) is a [batch, T] vector of scalar point anomaly scores produced before GAP, or a [batch, T, d] tensor of temporal embeddings. If the latter, compute L_smooth as written in the paper (elementwise squared difference of scalars) and check whether it crashes or is silently replaced by a norm. Also confirm the same tensor is used to rank points in Stage-2. If a scalar point-score tensor exists and is used consistently, the concern is resolved and only a notation fix is needed; if not, the point-level pipeline rests on an undefined quantity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step for the central point-level claim is the existence of the T-dimensional point-level anomaly scores h(X). Section 4.2.3 says \"the previous layer is the T-dimension output h(X), which can be seen as the anomaly scores of each point within the T-length segment,\" and Eq. (6) penalizes (h(X_i)_j - h(X_i)_{j+1})^2. But Eq. (2) constructs the segment representation as X = GAP(h_1,...,h_T) in R^d, and the only classifier output defined is the segment-level score f(X) = sigma(w^T X). No equation maps h_t to a scalar point score, and if h(X_i)_j denotes the j-th d-dimensional temporal embedding, the squared difference in Eq. (6) is not a scalar loss as written. Stage-2 also requires \"the anomaly score of each point\" to rank points, but the text never states where that per-point score comes from. Without a defined h(X), the TC loss and the point-level ranking have no well-defined object, so the reported point-level F1 and PA%K results are not reproducible from the paper's description alone. The released code may resolve this, but the paper as written does not specify the quantity on which the central point-level mechanism operates.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper considers point-level anomaly detection in multivariate time series when training data has only segment-level labels and a known fraction of positive segments is missing (treated as unlabeled). It proposes NRdetector, a two-stage framework: Stage-1 performs confidence-based sample selection (reliable-negative extraction plus label propagation), then trains a segment classifier with a loss that combines a proposed PU risk estimator and a temporal smoothness/separability term; Stage-2 ranks points inside predicted positive segments and uses an anomaly-rate estimator to set a threshold. Experiments on EMG, SMD, PSM, MSL, and SMAP compare against 13 baselines under several metrics, with ablations of the sample selector, the PU criterion, and the temporal embedding backbone. The central claims are that the proposed PU loss plus time-constraint loss generalizes across noise rates and that the full system outperforms all baselines in point-level F1 and PA%K F1 on all five datasets.","tokens_in":21799,"tokens_out":5007,"duration_ms":53045,"significance":"The problem formulation is practical and timely: real-world anomaly labels are often segment-level and noisy, while deployment requires point-level predictions. The paper ships public code and evaluates on five real-world benchmarks with a broad set of metrics, which is a strength. The proposed two-stage design, combining sample selection, PU learning, and a temporal-constraint loss, is a plausible architecture for this setting. However, the significance is currently limited by two load-bearing gaps: the claimed PU risk estimator in Eq. (5) is not derived and does not match the cited nnPU objective, and the point-level anomaly score h(X) used by the time-constraint loss and Stage-2 is not defined from the architecture in Eq. (2). If these are repaired with a precise derivation and a clear per-point scoring definition, the empirical comparison could be valuable; as written, the central mechanism is not reproducible from the text alone.","major_comments":[{"comment":"Equation (5) is presented as the Non-negative PU Risk Estimator and is justified by an upper-bound result from [62], but it does not match the nnPU objective: the nnPU risk has the form pi_P E_P[ell(f(X),1)] + max(0, E_U[ell(f(X),0)] - pi_P E_P[ell(f(X),0)]) for a surrogate loss ell, whereas Eq. (5) averages the sigmoid output f(X) itself and takes absolute deviations from 1 and pi_P. No derivation links the two, and the cited proposition from [62] is not stated. Because R_pu is the main segment-level objective in Eq. (9), the theoretical basis of Stage-1 is currently asserted rather than established.","section":"Section 4.2.3, Eq. (5)"},{"comment":"The point-level anomaly score h(X) used by Eq. (6) and by Stage-2 is never defined. Equation (2) builds a segment representation X = GAP(h_1,...,h_T) in R^d and produces one scalar S = sigma(w^T X); no equation specifies a T-dimensional scalar score per segment. If h(X_i)_j in Eq. (6) is the j-th d-dimensional temporal embedding, then the squared difference is not a scalar loss; if it is a scalar point score, the text does not say how it is computed from the pooled representation. Since L_smooth and the Stage-2 ranking both require per-point scores, the central point-level mechanism has no well-defined object as written. Please specify h(X) precisely, for example as a per-timestep output head applied before pooling, and correct Eq. (6) accordingly.","section":"Section 4.2.3, Eq. (6), and Section 4.3"},{"comment":"The claim that Stage-2 automatically determines the threshold is only partially supported: Table 10 fixes anomaly_ratio per dataset, and Section 4.3 describes k as a user-specified hyperparameter. Similarly, the class prior pi_P in Eq. (5) is set per dataset in Table 10. No validation or selection protocol for these per-dataset values is described, so the pipeline is not fully self-contained and the 'training-free automated estimator' does not remove the dependence on the fitted k. At minimum, the paper should report sensitivity to k and pi_P on all datasets or justify fixed values, and should clarify what the HOC estimator actually estimates given k.","section":"Section 5.2, Table 10, and Section 4.3"},{"comment":"All experimental tables report single point estimates with no standard deviations, confidence intervals, or number of seeds. This matters for the headline claim: on SMD in Table 2, NRdetector's F1 (0.1092) is close to WETAS (0.1020) and TreeMIL (0.0999), and in Table 4 under noise rate 0.0 the ranking is not consistently first. Without repeated-run statistics, the statement that NRdetector achieves the best results under the pure F1 score on all benchmark datasets is not statistically grounded.","section":"Tables 2-4 and Section 5.3"}],"minor_comments":[{"comment":"The symbol S in Eq. (2) is a segment-level score, yet Section 4.3 refers to 'anomaly scores S corresponding to each point'; the notation should be separated to avoid ambiguity.","section":"Section 4.2.1"},{"comment":"The text says 'NRdetector still outperforms WEATS and TreeMIL' where the intended baseline name is WETAS; please correct this typo.","section":"Section 5.3"},{"comment":"The caption says 'Aff-P and Aff-P are the precision/recall pair' but the column names are Aff-P and Aff-R; the caption should be corrected.","section":"Table 3 caption"},{"comment":"The hyperparameter is listed as 'anoamly_ratio' in the table; this should be 'anomaly_ratio'.","section":"Table 10"},{"comment":"The quantities Omega, Phi, and mu in Eq. (3) are not defined in the main text; the Katz index is also cited but not defined, so the sample selector is hard to reproduce from the paper alone.","section":"Section 4.2.2, Eq. (3)"}],"recommendation":"major_revision","confidential_remarks":"The reader's concern about the undefined point-level score is, on inspection, accurate and central: the time-constraint loss and Stage-2 ranking both depend on per-point scores that the architecture description does not provide. The code release may resolve this, but the paper must state the construction explicitly. Similarly, Eq. (5) should either be derived or replaced with the actual nnPU objective used in the code. These are fixable in a revision, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: this is a sensible engineering combination for a real problem—point-level anomaly detection from noisy segment-level labels—and the paper carries its weight empirically. But the formal core has a hole. The TC loss in Eq. (6) and the Stage-2 ranking both depend on a T-dimensional per-point anomaly score h(X), and the paper never defines where that score comes from. Eq. (2) pools the temporal embeddings h_t into a single segment vector X = GAP(h_1,...,h_T) and then produces a segment-level score f(X) = sigma(w^T X). Section 4.2.3 says 'the previous layer is the T-dimension output h(X)', but that's not the h_t from Eq. (2), and no mapping from h_t to scalar point scores is given. The stress-test note is right: as written, the TC loss and the point ranking have no well-defined object.\n\nWhat's genuinely new: the combination of a PU risk estimator with a temporal-constraint loss (smoothness plus separability) and a HOC-based automatic threshold is not in the cited prior work, as far as I can tell. The empirical study is broad: five multivariate benchmarks, 13 baselines, multiple metrics, ablations, and the code is released. The ablation tables suggest each component contributes. On the headline numbers, they beat WETAS and TreeMIL on most datasets at high label noise, which is a meaningful result if it holds.\n\nThe soft spots, in proportion: (1) Eq. (5) is asserted as a non-negative PU risk but doesn't match the cited nnPU objective and has no derivation; (2) the h(X) gap I mentioned; (3) Table 10 shows per-dataset priors and anomaly ratios that look tuned to each dataset, with no stated validation protocol; (4) no error bars anywhere, so the 'consistently outperforms' claim rests on single runs. None of these is fatal by itself—the empirical claim may survive a re-run with proper protocol—but together they mean the paper as written is not reproducible from the text alone. The code might resolve the h(X) question, but the text doesn't.\n\nAs a colleague: read this as a well-motivated systems paper with a promising practical setting, not as a finished formal method. If I were handling it, I'd send it to referees with a request for major revision, specifically to define h(X), correct or derive the PU loss, and add error bars with a validation protocol. It deserves serious review; the core idea is worth engaging with.","headline":"A practical PU-learning pipeline for point-level TSAD from weak segment labels, with broad experiments but a central formal gap: the per-point score h(X) is undefined and conflicts with the pooling in Eq. (2).","tokens_in":22355,"tokens_out":3315,"would_cite":false,"duration_ms":33110,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Point-level anomalies in time series can be localized from coarse, noisy segment-level labels by combining positive-unlabeled learning with a temporal smoothness and separability loss, and the resulting framework reports the best F1 among…","keywords":["time series anomaly detection","weakly supervised learning","positive and unlabeled learning","noisy labels","segment-level labels","point-level anomaly detection","temporal constraint loss","multivariate time series"],"falsifier":"Open the released code and trace the tensor $h(\\cdot)$ in Eq. (6): if it is the global-average-pooled segment vector broadcast across the $T$ positions instead of per-time-step outputs, the smoothness term is acting on an artifact and the claimed bridge from segment labels to point labels collapses.","tokens_in":21311,"feed_emoji":"🎯","tokens_out":8795,"duration_ms":86371,"temperature":0.7,"pith_summary":"This paper tries to establish that point-level anomalies in multivariate time series can be detected from weak, noisy segment-level labels—labels that say an anomalous event occurred somewhere in a window, but not where, with most windows entirely unlabeled. It proposes NRdetector, a two-stage framework: a coarse PU-learning stage that trains a segment classifier on labeled positives plus confidently selected negatives, regularized by a loss encouraging smooth per-point anomaly scores and separation between positive and negative segments; then a fine-grained stage that ranks points inside predicted positive segments and thresholds them with an estimated anomaly rate. The authors report that NRdetector beats 13 weakly, semi-, and unsupervised baselines under both pure F1 and PA%K F1 on all five real-world datasets (EMG, SMD, PSM, MSL, SMAP), with the largest margins at high label-noise rates. If true, the practical payoff is that cheap event-level annotation, not expensive point-level annotation, could be enough to localize anomalies.","feed_headline":"Noisy segment labels yield sharp point-level anomaly detection","feed_subtitle":"NRdetector combines PU learning with a time-constraint loss to localize anomalous points from weak segment labels.","key_machinery":"The load-bearing machinery is the training objective, not a single architectural novelty. Stage-1 uses the non-negative PU risk estimator $R_{pu}$ to classify segments, together with the Time-Constraint loss $\\mathcal{L}_c = \\lambda_1\\mathcal{L}_{\\text{smooth}} + \\lambda_2\\mathcal{L}_{\\text{sep}}$: $\\mathcal{L}_{\\text{smooth}}$ penalizes squared differences between anomaly scores of adjacent points inside a segment, and $\\mathcal{L}_{\\text{sep}}$ pushes the mean score of unlabeled (treated-negative) segments below that of labeled positives. Before training, a sample selector builds a Katz-index similarity graph over segments, extracts reliable negatives far from labeled positives, and propagates labels to clean the unlabeled set. Stage-2 then treats each predicted positive segment as a bag of $T$ points, sorts their per-point anomaly scores, and uses a clusterability-based anomaly-rate estimator to set the cutoff automatically. The paper argues that the PU risk upper-bounds the true classification risk and that reducing label noise tightens the generalization bound, which is what makes the pipeline noise-resilient.","core_discovery":"On its own terms, the paper's discovery is that the information gap between noisy segment-level labels and missing point-level labels can be bridged by combining a non-negative PU risk estimator with a time-constraint contrastive loss and a data-centric point selection stage. The PU loss treats known anomalous segments as positives and the rest as unlabeled rather than normal, which prevents the classifier from overfitting to hidden anomalies inside the unlabeled set. The time-constraint loss adds two priors to the per-point anomaly scores: temporal smoothness (adjacent points have similar scores) and separability (positive segments score higher than negative ones), giving the missing point-level supervision a proxy objective. After segment classification, Stage-2 ranks the points of every predicted positive segment and uses a training-free anomaly-rate estimator to pick the threshold, yielding point labels. Across EMG, SMD, PSM, MSL, and SMAP with segment label noise up to 0.6, the paper reports the best pure F1 and PA%K F1 among all compared methods, and its ablation studies attribute the gain to the combination of the PU loss, the TC loss, and the sample selector.","pith_inferences":["The smoothness prior is a bet against point outliers: anomalies that are isolated single-point spikes or abrupt jumps would be penalized by $\\mathcal{L}_{\\text{smooth}}$, so on datasets dominated by point-wise outliers the method may underperform a model without the TC loss.","The static anomaly-rate estimator in Stage-2 assumes the anomaly proportion is roughly constant; in non-stationary or streaming settings, an adaptive rate estimator would be needed to keep the threshold valid.","The same 'positive segment plus unlabeled bag plus temporal smoothness' recipe transfers naturally to video anomaly localization or audio event detection, where segment-level positive labels and unlabeled data are the standard annotation format.","If the released code shows that $h(\\cdot)$ in the TC loss is the pooled vector broadcast to length $T$ rather than true per-time-step outputs, then the point-level supervision is an artifact and the reported gains would need re-attribution to segment-level regularization only."],"forward_implications":["Point-level detection becomes feasible with event-level annotation: a domain expert only marks that an anomaly occurred within a window, and the model localizes it, cutting labeling cost.","Unlabeled data no longer need to be treated as clean normal data; hidden anomalies inside the unlabeled set are handled as PU noise rather than corrupting a normality model.","Every component downstream of the temporal embedding—sample selection, PU loss, TC loss, anomaly-rate threshold—is separable, so improvements in any one can be swapped in without redesigning the rest.","Lowering the segment-level label noise rate (labeling more true positive segments) tightens the paper's generalization-error upper bound, giving a concrete return on annotation investment.","The 11-metric evaluation positions NRdetector as a reference for weak-label time series anomaly detection comparisons, moving beyond pure F1 to PA%K, affiliation, and VUS scores."],"supporting_citations":[{"why":"Supplies the temporal embedding extractor the framework is built on and serves as the primary weakly supervised baseline.","marker":"[22]"},{"why":"Provides the network-based PU assumptions (label propagation and reliable-negative selection) used by the sample selector.","marker":"[8]"},{"why":"Supplies the Katz-index similarity measure used to build the KNN graph for extracting reliable negatives.","marker":"[32]"},{"why":"Contributes the non-negative PU risk estimator that forms the PU loss in Stage-1 robust segment learning.","marker":"[19]"},{"why":"Provides the distribution-alignment reformulation and the upper-bound result the paper uses to justify optimizing the PU risk.","marker":"[62]"},{"why":"Supplies the training-free anomaly-rate estimator (HOC) used to automate the point-level threshold in Stage-2.","marker":"[68]"},{"why":"Is the strongest recent weakly supervised MIL baseline that NRdetector must outperform.","marker":"[27]"},{"why":"Is the DeepMIL baseline representing multiple-instance learning approaches to segment-supervised anomaly detection.","marker":"[44]"}],"fun_headline_variants":["PU learning and time-constraint loss localize anomalies from weak segment labels","NRdetector turns noisy segment labels into precise point anomalies","Weak segment labels, sharp point detection: NRdetector bridges the gap","Noise-resilient point detection from segment labels via NRdetector","NRdetector: PU learning plus time-constraint loss for point anomalies"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The training losses assume the model produces a real anomaly score for every time point inside a segment, even though the architecture pools the temporal embeddings into one vector before classification.","fun_headline_variants_meta":{"raw":{"variants":["PU learning and time-constraint loss localize anomalies from weak segment labels","NRdetector turns noisy segment labels into precise point anomalies","Weak segment labels, sharp point detection: NRdetector bridges the gap","Noise-resilient point detection from segment labels via NRdetector","NRdetector: PU learning plus time-constraint loss for point anomalies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00139,"raw_usage":{"total_tokens":5648,"prompt_tokens":991,"completion_tokens":4657,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":607,"completion_tokens_details":{"reasoning_tokens":4562}},"tokens_in":607,"tokens_out":4657,"duration_ms":32300,"temperature":1.0,"reasoning_tokens":4562,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:39:57.899901+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Open the released code and trace the tensor $h(\\cdot)$ in Eq. (6): if it is the global-average-pooled segment vector broadcast across the $T$ positions instead of per-time-step outputs, the smoothness term is acting on an artifact and the claimed bridge from segment labels to point labels collapses.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the temporal embedding extractor the framework is built on and serves as the primary weakly supervised baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the network-based PU assumptions (label propagation and reliable-negative selection) used by the sample selector."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Katz-index similarity measure used to build the KNN graph for extracting reliable negatives."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contributes the non-negative PU risk estimator that forms the PU loss in Stage-1 robust segment learning."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the distribution-alignment reformulation and the upper-bound result the paper uses to justify optimizing the PU risk."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the training-free anomaly-rate estimator (HOC) used to automate the point-level threshold in Stage-2."},{"cited_title":"TreeMIL: A Multi-instance Learning Framework for Time Series Anomaly Detection with Inexact Supervision","cited_arxiv_id":"2401.11235","evidence_quote":"Is the strongest recent weakly supervised MIL baseline that NRdetector must outperform."},{"cited_title":"PeerJ Computer Science 7 (2021), e757","cited_arxiv_id":null,"evidence_quote":"Is the DeepMIL baseline representing multiple-instance learning approaches to segment-supervised anomaly detection."}],"review_version":1}