{"id":"678ecf9a-32ce-4d05-a8d0-8daa0ee97aa9","arxiv_id":"2607.13368","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A pipeline that screens STFT settings by Silhouette score, trains one autoencoder per severity class, and rejects via latent-box plus reconstruction-error checks reports H-scores of 0.9924 (CWRU) and 0.9509 (PU) for fine-grained open-set fault severity diagnosis.","lead":"This paper presents a bearing-fault detector that ranks spectrogram settings by how well known fault states separate, then gives each severity level its own small autoencoder and rejects anything that fails two separate checks. On two standard bearing benchmarks it reports open-set H-scores of 0.9924 and 0.9509 and claims it found the best settings while checking only 9 of 38 and 2 of 39 candidates.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MGDC efficiency claim is confounded: Silhouette ranking tracks frequency-bin count, and R=9/R=2 are read from exhaustive curves, so 'metric-guided' may add nothing over a max-bin sweep.","rationale":"The reported H-scores are the paper's strongest, best-supported claim. The architecture is plausible: a held-out test set, selection-set-based hyperparameter choice, ablation tables (13-14), and correct union-bound arithmetic (§3.5.1). The MGDC contribution, however, is the paper's headline data-centric innovation and the only source of the 5.69x/29.87x cost reductions. The Tables 4/9 evidence indicates the Silhouette ranking is dominated by frequency-bin count, and the R values are determined post hoc from exhaustive-search curves. This is a load-bearing weakness because the abstract's second sentence is a claim about selection efficiency, not just about the diagnostic architecture. The reader's weakest_assumption already identifies the proxy issue; my analysis adds the specific confound (bin-count monotonicity) and the hindsight-R problem. The concrete test above would settle it. If the test shows a max-bin baseline matches Silhouette, the MGDC claims should be re-scoped or removed, but the CSAE architecture results can stand; hence the verdict remains CONDITIONAL rather than REJECT. I set UNCHANGED because my concern does not alter the reader's overall verdict.","tokens_in":57631,"tokens_out":12965,"duration_ms":120403,"concrete_test":"On each dataset, construct a baseline ranking π_bins that sorts the 38/39 STFT candidates by descending frequency-bin count (ties: descending signal length), and compute its cumulative best H-score on the selection set for R=1..38 (or 39). Compare with the Silhouette-ranked curves in Figs 7/13. If π_bins reaches the global-best H-score at the same or smaller R (9 on CWRU, 2 on PU) and with the same value (0.9948/0.9883), Silhouette is a proxy for bin count and the MGDC claim fails. Additionally, pre-register a fixed R0=5 before inspecting the exhaustive curves and report the best H-score among the Silhouette top-5; if it is materially below the exhaustive best, the reported R=9/R=2 are hindsight-chosen.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central efficiency claim — MGDC identifies the exhaustive best while validating only 9/38 (CWRU) or 2/39 (PU) candidates — depends on the Silhouette score being a faithful proxy for downstream open-set H-score. Tables 4 and 9 show this proxy is confounded by frequency-bin count. Within every signal length, Silhouette decreases almost monotonically as bins drop from 2048 to 16 (CWRU 3-rev: 0.1593→0.0407), whereas H-score is non-monotone (0.8615, 0.9844, 0.9948, 0.6943, ...). The global-best CWRU configuration (3-rev, 512 bins, 16 time steps) has Silhouette 0.1277 and is only the 9th-ranked candidate because four 2048-bin and four 1024-bin candidates rank ahead of it. On PU the best configuration is a 2048-bin candidate ranked 1-2. Thus R=9 and R=2 are not outputs of MGDC; they are the ranks at which the best frequency-bin class first enters the candidate set, read from the exhaustive-search curves in Figs 7 and 13. Section 3.3 specifies no a priori stopping rule for R, so the 5.69x/29.87x speedups are retrospective, not guaranteed for a pre-specified budget. A trivial ranking by descending frequency-bin count would likely reproduce the same cumulative-best-H-score curves, meaning the Silhouette screen adds no information beyond a heuristic frequency-resolution sweep. The paper itself admits top-1 selection is unstable (Sec. 4.5.2), but that does not rescue MGDC: the method selects h* by H-score on HR, and the question is whether the choice of HR has any metric-guided content.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a two-stage method for fine-grained open-set fault diagnosis of rotating machinery. The first stage, Metric-Guided Data-Centric (MGDC) STFT configuration selection, ranks predefined time-frequency resolutions by the Silhouette score computed on flattened, normalized spectrograms, retains the top-R candidates, and picks the one with the highest H-score on a selection set. The second stage trains a shared lightweight CNN feature extractor and a bank of class-specific autoencoders (CSAEs); classification is by minimum reconstruction error, and rejection of unknown severities uses a conjunction of element-wise latent quantile bounds and a reconstruction-error quantile threshold. Experiments on CWRU and PU report H-scores of 0.9924 and 0.9509 for fine-grained severity diagnosis, with large margins over Global AE, OpenMax, CPL, and ARPL, and claim that MGDC identifies the exhaustive-search-best STFT configuration after validating only 9/38 or 2/39 candidates (speedups of 5.69x and 29.87x).","tokens_in":57901,"tokens_out":7360,"duration_ms":70808,"significance":"If validated, the CSAE bank plus dual-criteria rejection is a valuable recipe for severity-level OSFD. The gains are large and consistent across datasets: e.g., CWRU severity H-score 0.9924 vs 0.8015 (Global AE) and 0.4449 (CPL); PU severity 0.9509 vs 0.8822 (CPL). The evaluation protocol is careful: disjoint 0.6/0.1/0.1/0.2 train/validation/selection/test splits, selection-set-based hyperparameter choice, ablations, a threshold-sensitivity analysis, and explicit limitations. The math in Section 3.5.1 is correct, and the class-specific vs global threshold ablation shows that class-specific calibration matters. However, the MGDC efficiency claim is not presently established: the Silhouette screen appears confounded with frequency-bin count, and R is chosen retrospectively from exhaustive-search curves. The diagnostic architecture may stand on its own, but the abstract and Section 4.7 overstate the preprocessing contribution.","major_comments":[{"comment":"The central efficiency claim is not supported as a metric-guided property. In Tables 4 and 9, Silhouette decreases almost monotonically with frequency-bin count within every signal length (CWRU 3-rev: 0.1593 at 2048 bins down to 0.0407 at 16 bins), whereas H-score is non-monotone (0.8615, 0.9844, 0.9948, 0.6943, ...). On CWRU the best configuration (3 rev, 512 bins, 16 steps) is ranked 9th by Silhouette only because eight 2048/1024-bin candidates precede it; on PU the best configuration is a 2048-bin candidate ranked near the top. This is exactly what a ranking by descending frequency-bin count would predict, so the Silhouette computation may add no information beyond a frequency-resolution heuristic. The paper should include a control baseline—e.g., ranking by bin count alone or a random ranking—and show that Silhouette tracks H-score better than such a baseline. Without that, the phras","section":"§3.3; Tables 4 and 9; Figs. 7 and 13"},{"comment":"R is a free parameter chosen after seeing the exhaustive-search curves. Section 3.3 says candidates are sorted and the top R retained, but gives no rule for choosing R; Figs. 7 and 13 then report that the cumulative-best H-score reaches the global best at R=9 (CWRU) and R=2 (PU). These R values are therefore read off the ground-truth curves, not produced by MGDC under a pre-specified budget. The speedup factors in Table 12 and the abstract ('evaluating only 9 of 38 ... 2 of 39 ... reducing selection cost by factors of 5.69 and 29.87') are retrospective. To support the claim, the authors should either fix R (or a validation budget) in advance and report the resulting performance, or explicitly reframe MGDC as an anytime heuristic whose reported R values are illustrative post-hoc examples.","section":"§3.3, §4.5.2, §4.6.2, Table 12"}],"minor_comments":[{"comment":"No repeated trials or random seeds are reported. The severity-task margins are large, so the qualitative conclusions are likely robust, but standard deviations would help, especially for the PU type-level comparison (H-score 0.8698 vs CPL 0.9583).","section":"§4.1, §4.3"},{"comment":"The TPE/Hyperband/BOHB comparison lacks budget details (number of trials, early-stopping thresholds, seed handling). Without these, the 'Speedup' column is hard to interpret fairly.","section":"§4.7, Table 12"},{"comment":"The H-score distributions at α=0.9999 are wide across STFT configurations. Reporting the spread (or at least min/max) would strengthen the sensitivity analysis and support the conclusion that configuration quality matters.","section":"§4.8, Fig. 19"},{"comment":"The union-bound false-rejection rate ≤ 2dz(1−α) is correct for population quantiles. Since thresholds are estimated from a finite training set, actual coverage may differ; the paper could note that the bound is asymptotic or conditional on the estimated thresholds.","section":"§3.5.1"},{"comment":"Minor notation inconsistency: 'LetS∈R^{M×B}' should be 'Let S∈...' with proper math formatting. Also, 'Section 3 addresses these three problems' is slightly abrupt; adding a short roadmap would improve readability.","section":"§2.2"}],"recommendation":"major_revision","confidential_remarks":"The main barrier is the MGDC efficiency claim. The stress-test concern lands: Silhouette ranking is confounded with frequency-bin count, and R=9/R=2 are retrospective. The diagnostic architecture and evaluation are otherwise solid. I would be willing to accept a revised version that adds a bin-count control baseline, pre-specifies or redefines R/budget, and tempers the abstract's efficiency claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe bottom line: the class-specific autoencoder bank with dimension-wise latent bounds and reconstruction-error rejection is a credible, well-tested recipe for fine-grained severity-level open-set diagnosis. The reported H-scores (0.9924 CWRU, 0.9509 PU) are far above the baselines, and the ablations back up the design choices. But the MGDC efficiency claim is weaker than the abstract suggests: the Silhouette ranking is effectively a frequency-bin-count sweep, and R=9/R=2 are hindsight numbers.\n\nWhat is actually new: severity-level (not just type-level) open-set tasks on two standard bearing benchmarks; a clean 0.6/0.1/0.1/0.2 split with hyperparameters (alpha, dh/dz, STFT config) selected on a selection set; an ablation study that shows the latent-boundary criterion carries most of the rejection power and that class-specific thresholds matter; and a correct union-bound calculation for the quantile calibration. That part is reproducible in principle and honestly done.\n\nThe soft spot is in Tables 4 and 9. Within every signal length, Silhouette falls almost monotonically as frequency bins drop from 2048 to 16. On CWRU the globally best configuration (3 rev, 512 bins) is ranked 9th because eight 2048/1024-bin candidates rank ahead of it; a trivial descending-bin sweep would give the same top-9. On PU the best configuration is a 2048-bin one and sits in the top-2. So the \"metric-guided\" part seems to add no information beyond \"use many bins, then try a few lower-bin options.\" The paper never tests against a simple max-frequency-bin rule, and Section 3.3 gives no a priori stopping rule for R; the 5.69x/29.87x speedups are read from the exhaustive curves. That is an overstatement, though not necessarily a fatal one: the selected configuration is still good, and the anytime curves are honestly presented. Top-1 instability is acknowledged.\n\nSecondary issues: no code/data artifacts, single runs without seed variance, and the PU operating-condition setup is vague.\n\nThis paper is for PHM researchers who want a practical severity-level OSFD pipeline. The CSAE part is worth citing and building on; the MGDC part needs to be re-scoped or re-analyzed. I would send it to a serious referee, but the referee should ask for a max-bin ablation, a pre-specified budget, and multi-seed results.","headline":"The CSAE-based severity open-set pipeline is credible and worth engaging, but the MGDC efficiency claim is overstated: the Silhouette screen behaves like a frequency-bin sweep and R=9/R=2 are hindsight, not a priori budgets.","tokens_in":58620,"tokens_out":4230,"would_cite":true,"duration_ms":42129,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-grained open-set fault diagnosis—separating unseen severity levels of a known fault type—is achievable with class-specific autoencoders and a Silhouette-score STFT screen, reaching H-scores of 0.9924 and 0.9509 on two bearing datasets.","keywords":["open-set fault diagnosis","class-specific autoencoder","fine-grained severity diagnosis","STFT configuration selection","Silhouette score","anomaly rejection","bearing fault diagnosis","predictive maintenance"],"falsifier":"Take a third rotating-machinery dataset with an STFT search grid where the exhaustive-best H-score sits at a low-Silhouette configuration (Table 4 already shows such non-monotonicity), run MGDC with R fixed, and check whether the top-R Silhouette candidates ever include the exhaustive best; if they do not, the screening claim fails. A minimal version: replace the top-9/top-2 Silhouette picks on CWRU/PU with nine/two random configurations and ask whether the achieved best H-score is statistically indistinguishable.","tokens_in":57298,"feed_emoji":"⚙️","tokens_out":8012,"duration_ms":80797,"temperature":0.7,"pith_summary":"The paper targets a harder open-set problem than is usually studied: instead of deciding whether a signal comes from an unseen fault type, the model must decide whether it comes from an unseen severity level of a fault type it already knows. The authors argue that two changes make this tractable. First, they replace the customary trial-and-error choice of STFT parameters with a metric-guided screen: candidate time-frequency configurations are ranked before network training by the Silhouette score of their flattened normalized spectrograms, and only the top few candidates are validated by actual training. Second, they replace a single global autoencoder with a bank of class-specific autoencoders (CSAEs), one per known severity class, and accept a test sample as a known class only if both its latent coordinates and its reconstruction error are inside per-class quantile bounds. On the CWRU and PU bearing datasets the method reports severity-task H-scores of 0.9924 and 0.9509, and the screen recovers the exhaustive-search-best configuration after evaluating 9 of 38 candidates on CWRU and 2 of 39 on PU. If these results hold, tracking progressive degradation in an open-set setting becomes practical at modest configuration-selection cost.","feed_headline":"Autoencoder bank separates unseen bearing fault severities at H-score 0.95","feed_subtitle":"One autoencoder per severity class plus a Silhouette screen finds the best time-frequency settings at a fraction of exhaustive-search cost.","key_machinery":"The load-bearing machinery is the bank of class-specific autoencoders (CSAE)—K independent encoder-decoder pairs sharing a three-layer lightweight CNN feature extractor, trained with an error-based softmax cross-entropy so that branch k reconstructs class k best—combined with a dual-criteria anomaly rejection rule: per-class element-wise α-quantile bounds on the latent vector and a per-class α-quantile threshold on the L1 reconstruction error, with AND-to-accept logic. The preprocessing-level machinery is MGDC: candidate STFT configurations are transformed by square-root scaling and per-frequency-bin min-max normalization, flattened, and scored by the Silhouette score (using known severity l","core_discovery":"The central claim is that fine-grained open-set fault severity diagnosis can be decomposed into two mechanisms that each address a known failure mode. The CSAE bank addresses the failure of global autoencoders: one autoencoder per known severity class learns a compact, class-specific manifold, and the cross-entropy over softmax-normalized negative reconstruction errors drives each branch to specialize. The dual-criteria rejection addresses the failure of scalar anomaly scores: instead of compressing the latent vector into one distance or probability, the method keeps an element-wise quantile boundary per latent dimension and combines it with a class-specific reconstruction-error threshold in","pith_inferences":["The Silhouette ranking in the paper's tables is nearly monotone in frequency-bin count while H-score is not (e.g., the 3-revolution row: Silhouette decreases 0.1593→0.0407 as bins drop 2048→16, but H-score peaks at 512 bins and falls at 256). This raises the possibility that MGDC is mostly sweeping frequency resolution rather than measuring true separability, and that R=9 and R=2 were chosen by lo","A falsifying-style extension: compare Silhouette-ranked top-R against R random configurations. If random subsets of the same size find the same best H-score, the metric adds no information beyond a resolution prior.","The element-wise latent quantile boundary is a nonparametric, coordinate-wise stand-in for distance-based out-of-distribution scores; it should transfer to other dense-manifold OOD problems, but its success here may depend on the latent dimension being deliberately small (dz=2), so scaling to higher dz needs its own study.","The paper's limitation section says MGDC uses known labels during screening; an unlabeled variant (e.g., clustering without label supervision) is the natural next test for real deployments where clean severity labels are not available at pre-processing time."],"forward_implications":["If the CSAE plus dual-criteria recipe is correct, a maintenance system can report both the fault type and the progression stage of a known degradation while flagging unseen stages, without collapsing to a scalar anomaly score.","MGDC makes STFT configuration selection an anytime procedure: on CWRU the top-1 candidate gives H-score 0.9323 in 147 s and the top-9 candidate matches the exhaustive-search best 0.9948 at a 5.69x cost reduction; on PU the top-2 candidate matches the global best 0.9883 at 29.87x reduction.","The same architecture remains competitive but not best-in-class for coarse fault-type OSFD on PU, where a distance-based baseline reaches a higher H-score; so the method's claimed advantage is specific to dense severity manifolds.","The dual-criteria rejection concentrates most of the gain in the latent-boundary criterion: ablations show latent-boundary-only already achieves H-scores of 0.9786 (CWRU severity) and 0.9469 (PU severity), while reconstruction-error-only or AND-reject variants drop sharply."],"fun_headline_variants":["Per-class autoencoders plus Silhouette-guided STFT hit H-score 0.99 on bearing faults","Class-specific autoencoders reject unknown fault severities at H-score 0.95","Metric-guided time-frequency picker plus per-class AEs: open-set severity done right","Silhouette score picks STFT config, per-class AEs handle unknowns: 0.99 H-score"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the Silhouette score computed on flattened, normalized spectrograms ranks STFT configurations in the same order as the downstream open-set H-score; if that ranking is mostly a frequency-resolution artifact, the claimed efficiency of MGDC collapses to a heuristic frequency-bin sweep.","fun_headline_variants_meta":{"raw":{"variants":["Per-class autoencoders plus Silhouette-guided STFT hit H-score 0.99 on bearing faults","Class-specific autoencoders reject unknown fault severities at H-score 0.95","Metric-guided time-frequency picker plus per-class AEs: open-set severity done right","Silhouette score picks STFT config, per-class AEs handle unknowns: 0.99 H-score"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001869,"raw_usage":{"total_tokens":7213,"prompt_tokens":826,"completion_tokens":6387,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":570,"completion_tokens_details":{"reasoning_tokens":6286}},"tokens_in":570,"tokens_out":6387,"duration_ms":37176,"temperature":1.0,"reasoning_tokens":6286,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T05:23:04.387843+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a third rotating-machinery dataset with an STFT search grid where the exhaustive-best H-score sits at a low-Silhouette configuration (Table 4 already shows such non-monotonicity), run MGDC with R fixed, and check whether the top-R Silhouette candidates ever include the exhaustive best; if they do not, the screening claim fails. A minimal version: replace the top-9/top-2 Silhouette picks on CWRU/PU with nine/two random configurations and ask whether the achieved best H-score is statistically indistinguishable.","supporting_citations":[],"review_version":1}