{"id":"8fc16d8d-e017-4a8a-b235-c0fe41532a9b","arxiv_id":"2502.06424","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"CS-SHAP explains neural-network fault diagnoses by running SHAP on cyclic-spectral maps, jointly attributing carrier and modulation frequencies.","lead":"A new post-hoc explanation method, CS-SHAP, applies SHAP to the cyclic-spectral domain of vibration signals, so explanations show both the carrier and modulation frequencies of a fault instead of only one. The method could make neural-network-based fault diagnosis more trustworthy in industrial settings where maintenance decisions must be justified.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The inverse CS transform's phase handling under SHAP masking is unspecified; masked CS samples are inverted with the explained sample's phase, so reconstructed signals likely have different CS content than intended, making the claimed attributions potentially artifact-driven.","rationale":"The paper makes a concrete, testable methodological contribution and includes code plus a simulation dataset with known fault logic, which is real support for the basic pipeline. The derivation of Z for deterministic signals is coherent, and for a single unperturbed sample Z^{-1}∘Z recovers the signal, assuming a valid STFT inverse. The gap is in the SHAP perturbation regime: Eq. (5) requires evaluating the integrated model on masked CS samples, but the phase needed by Eq. (18) is not specified and the reconstructed inputs are not guaranteed to have the intended CS content. This is exactly the load-bearing assumption behind the 'absence' and 'presence' attributions that CS-SHAP claims as its main advantage. I agree with the reader's identification of this weakness and sharpen it: the sqrt of a masked inverse FT may be undefined, and the STFT consistency constraint means the actual CS of reconstructed signals can differ from the masked CS. The proposed consistency test directly checks this. If it passes, the concern is resolved; if it fails, the central comparative claims are not supported. I would therefore retain a conditional verdict until this test is run.","tokens_in":18986,"tokens_out":6372,"duration_ms":58672,"concrete_test":"Instrument the released code on the simulation dataset: inside the SHAP expectation loop, after Z^{-1} reconstructs a time signal for a masked CS sample, apply Z to that reconstructed signal and compute ||Z(Z^{-1}(CS_masked, θ)) - CS_masked||_2 / ||CS_masked||_2 averaged over at least 100 masked draws. Also log the fraction of (f,α) bins for which IFFT(CS_masked) is negative, where sqrt is undefined. If the relative reconstruction error is above a small tolerance (e.g., 1%) or the negative-bin fraction is nonzero, the integrated model is not being explained by the claimed CS features, and CS-SHAP's empirical comparisons would need to be revisited.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central empirical claim—that CS-SHAP yields 'clearer and more accurate' attributions—requires that each SHAP subset evaluation in Eq. (5) evaluates the true end-to-end model on a signal whose CS representation is the masked CS sample. This is not established. Z^{-1} in Eq. (18) reconstructs STFT from sqrt(∫ CS_x(f,α) e^{i2παt} dα) and a phase θ, but Sections 3.2 and 3.3 never specify which θ is used for masked CS samples produced by SHAP's background substitution. If, as Fig. 3 and the sample preprocessing suggest, θ is taken from the explained sample, then the reconstructed time signal is a hybrid: its magnitude comes from a mixture of explained and background CS values, while its phase comes from the explained sample. After iSTFT, the true CS of this synthetic signal will not equal the masked CS, because STFT magnitude and phase are coupled through the redundancy of the STFT frame; the model is therefore evaluated on off-manifold inputs that do not correspond to any real CS feature configuration. If instead θ is drawn from a background sample, the phase and magnitude are mismatched and the same problem recurs. Additionally, the inverse FT of a masked CS need not be non-negative, so sqrt in Eq. (18) can fail or require ad-hoc clamping. Consequently, the 'presence/absence' attributions in Figs. 6–19 may not be Shapley values of the CS representation at all; they may be artifacts of the reconstruction rule.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CS-SHAP, a post-hoc explanation method that applies SHAP in the cyclic-spectral (CS) domain to fault-diagnosis classifiers. It derives a CS transform Z and an inverse transform Z^{-1} for deterministic signals, integrates Z^{-1} with a trained end-to-end model, and computes SHAP attributions over CS-domain features. The authors claim that this yields explanations in terms of both carrier frequency f_c and modulation frequency f_m, aligning with fault mechanisms and outperforming time-, frequency-, time-frequency-, and envelope-domain SHAP. Experiments are reported on a simulation dataset, the CWRU bearing dataset, and a private helical gearbox dataset.","tokens_in":19350,"tokens_out":6167,"duration_ms":52223,"significance":"The idea of extending SHAP to the CS domain is timely and potentially useful for rotating-machinery fault diagnosis, where cyclostationary signatures are highly diagnostic. The paper's strengths include a clear problem statement, open-source code, and a simulation dataset with predefined fault logic that provides an explicit ground truth for one experiment. For a single sample, the forward transform identity in Eq. (13) is correct, and the inverse in Eq. (18) is mathematically consistent when the phase is exact. However, the central empirical claim of \"clearer and more accurate explanations\" is not yet substantiated: the real-data validation is partly circular, the comparisons are qualitative, and the phase-handling procedure under SHAP masking is unspecified. If the phase issue is resolved and quantitative evaluation is added, the method could become a useful benchmark for IFD interpretability.","major_comments":[{"comment":"The paper does not specify how the phase θ_x(f,t) is obtained when SHAP creates masked CS samples by substituting background values. If the phase of the explained sample is used, as Fig. 3(b) suggests, the reconstructed time signal after iSTFT is a hybrid whose true CS representation does not equal the masked CS; the integrated model is then evaluated on off-manifold inputs, so the computed quantities are not Shapley values over the CS features of the original model. If the phase is taken from a background sample, phase and magnitude are mismatched in the same way. The authors should state the phase policy, prove that the reconstruction preserves the masked CS, or provide evidence that the attributions are insensitive to the phase choice.","section":"Section 3.2-3.3, Eq. (18)"},{"comment":"The inverse transform contains sqrt(∫ CS_x(f,α) e^{i2παt} dα), which is only well-defined when the integral is real and non-negative. For arbitrary masked CS inputs, which are mixtures of explained and background CS magnitudes, this condition can fail, and no clamping or projection is described. This is not a purely theoretical edge case: it is the operating regime of every SHAP subset evaluation. The paper needs to explain how the square root is computed numerically and whether the resulting reconstructed signal's CS representation equals the intended masked CS.","section":"Eq. (18)"},{"comment":"The ground-truth fault components for the CWRU and gearbox datasets are identified by visual inspection of the CS representations themselves (e.g., the annotations in Fig. 10(a) and the corresponding statements in Section 4.2), and CS-SHAP is then judged against these CS-derived annotations. This is partially circular. The simulation dataset in Section 4.1 has predefined logic and is therefore the strongest evidence, but the real-data claims should be supported by independent annotations (e.g., bearing characteristic frequencies computed from geometry and speed, or envelope-spectrum peaks) and by quantitative agreement measures.","section":"Section 4.2 and 4.3"},{"comment":"The comparisons are qualitative and based on a single sample per class, with no error bars, no multiple runs, and no numeric metric for explanation quality. The paper's own conclusion acknowledges that validation \"relies on subjective human judgment.\" To support the abstract's claim of \"clearer and more accurate explanations,\" the authors should report quantitative metrics (e.g., detection rate of known fault components, overlap with ground-truth regions, or rank correlation with fault logic) averaged over multiple samples and random seeds.","section":"Section 4 and 5"},{"comment":"The novelty claim that the authors are \"the first to rigorously derive and formally prove\" the CS transform is overstated. Equation (13) is the standard definition of cyclic spectral correlation applied to a deterministic signal, and the derivation follows directly from the Wiener-Khinchin relation in Eq. (12); the paper even cites Literature [41] as having implicitly used the same construction. The contribution should be reframed as an application of known CS analysis to SHAP, rather than a new transform, unless the authors identify a specific technical distinction from existing CSC estimators.","section":"Section 3.1"},{"comment":"The paper does not define the feature partition in the CS domain (individual (f,α) bins vs. grouped components) nor the SHAP approximation algorithm used (exact enumeration, KernelSHAP, or another estimator). Since the CS domain is two-dimensional and high-dimensional, exact Shapley values over individual bins are computationally prohibitive; without this specification, the results in Figs. 6–19 cannot be reproduced or interpreted as Shapley values of a well-defined game.","section":"Section 3.2"}],"minor_comments":[{"comment":"There are typos such as \"achive\" (should be \"achieve\") and \"Negetive\" in the Fig. 3 caption; please proofread the text.","section":"Section 1"},{"comment":"The wording \"sinc signal\" should be \"sine signal\" in the discussion following Eq. (14).","section":"Section 3.1"},{"comment":"In step ③, the text says time-domain samples are transformed using the \"inverse CS transform Z^{-1}\", but the forward transform Z is intended.","section":"Section 3.3"},{"comment":"The STFT magnitude of a finite-window sine is not a constant K independent of t; it involves the window's Fourier transform evaluated at f ± f1. Consider revising this illustrative derivation to avoid an incorrect exact statement.","section":"Eq. (15)"},{"comment":"The text says \"P 1O and P 2O share the same modulation frequency fBPIO\"; the symbol should be fBPFO, consistent with Table 3.","section":"Section 4.2"},{"comment":"The phrase \"significantly impulses the CS-SHAP results\" should read \"significantly impacts the CS-SHAP results.\"","section":"Section 5.1"},{"comment":"The color maps and labels in the attribution figures are hard to read at the printed resolution; the positive/negative contribution color bars should be made consistent across panels and figure captions should define the units.","section":"Figures 6-19"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's contribution is closer to an engineering application of known CS analysis than a new theoretical result; the phase-handling gap and the circular validation are the main technical obstacles. The paper may fit an applied signal-processing or engineering-informatics venue, but the claims need to be scaled back and the empirical evaluation strengthened before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing: this is a legitimate but modest extension—SHAP applied in the cyclic-spectral domain, giving joint carrier/modulation attribution. The CS transform derivation is not new; Eq. (13) is the standard spectrogram-based cyclic spectrum estimate, and the authors themselves note [41] where it is implicit. What's new is the packaging: defining an inverse, integrating it with SHAP, and showing qualitative visualizations on three datasets.\n\nThe paper does some things well. It clearly motivates the dual fc/fm explanation, the simulation setup has known fault logic and shows CS-SHAP can separate components that other SHAP variants confound, and the code is open. For the IFD community, this is a useful practical tool.\n\nBut the central evidence is soft. All comparisons are qualitative, no error bars, no quantitative attribution metrics. The real-dataset 'ground truths' are read off CS maps, the same representation the method uses, so the validation is partially circular. Private gearbox data isn't available, which limits reproducibility despite the code.\n\nThe bigger technical concern is the phase under SHAP masking. Eq. (18) reconstructs STFT from CS magnitude and a phase θ, but the paper never says which θ is used when SHAP substitutes background CS values. If θ is taken from the explained sample, the reconstructed time signal is a hybrid magnitude from masked CS with that sample's phase; its true CS will not equal the masked CS, because the STFT magnitude and phase are coupled through the frame. If θ is background, phase and magnitude are mismatched. In either case, SHAP is not evaluating the model on a signal whose CS is the masked feature set—it is evaluating on off-manifold reconstructions. That does not make the Shapley values meaningless (they are still values of the integrated function), but it does mean the 'presence/absence' interpretations in Figures 6–19 may be artifacts of the reconstruction rule rather than faithful CS attributions. There is also a minor technical issue: the inverse FT of a masked CS does not have to be nonnegative, so the sqrt in Eq. (18) may need clamping, which is unmentioned.\n\nThe overclaim about being 'first to rigorously derive' the transform should be toned down; the authors themselves cite prior implicit use.\n\nBottom line: this is a plausible, potentially useful method for IFD interpretability, and it addresses a real gap. But as written, the evidence does not support the 'clearer and more accurate' benchmark claim. A serious referee could ask for explicit phase handling, quantitative metrics, synthetic validation where the reconstruction can be checked, and non-circular ground truths. I would send it to review rather than desk reject, but it needs work.","headline":"A useful but overclaimed extension of SHAP to the cyclic-spectral domain; the core idea is fine, but the phase handling under masking and circular validation undercut the central 'clearer and more accurate' claim.","tokens_in":19844,"tokens_out":3382,"would_cite":false,"duration_ms":30899,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CS-SHAP extends Shapley-value explanations into the cyclic-spectral domain so that a fault prediction is attributed jointly to the carrier frequency and the modulation frequency, matching the physical mechanism of rotating-machine faults.","keywords":["cyclic-spectral transform","SHAP","interpretable fault diagnosis","post-hoc explanation","carrier frequency","modulation frequency","cyclostationary signal","neural network interpretability"],"falsifier":"Run the outer-race experiment again but with background samples for SHAP drawn from a distribution that assigns random phase per masked CS coefficient, instead of using the phase stored in the sample being explained; if the resulting attribution map changes materially, the explanation depends on an untested phase convention rather than on signal content.","tokens_in":18793,"feed_emoji":"🔧","tokens_out":4651,"duration_ms":38196,"temperature":0.7,"pith_summary":"The paper tries to establish that a post-hoc explanation method for neural-network fault diagnosis can and should attribute predictions in the cyclic-spectral (CS) domain, where a vibration signal is represented by a carrier frequency $f_c$ and a modulation frequency $f_m$ at the same time. It derives a forward transform $\\mathcal{Z}$ and an inverse transform $\\mathcal{Z}^{-1}$ for deterministic signals, then feeds SHAP through the inverse transform so the end-to-end network is still the object being explained. On a simulated dataset with known fault logic, on an open bearing benchmark, and on a gearbox dataset, the authors argue that CS-SHAP yields contributions that align with fault mechanisms and distinguishes components that share one frequency but differ in the other.","feed_headline":"Fault explanations now name both carrier and modulation frequencies","feed_subtitle":"A SHAP extension into the cyclic-spectral domain attributes vibration-fault decisions to $f_c$ and $f_m$ together.","key_machinery":"The load-bearing object is the cyclic-spectral transform pair. The forward transform $\\mathcal{Z}$ maps a deterministic signal $x(t)$ to $(\\mathrm{CS}_x(f,\\alpha), \\theta_x(f,t))$, where the CS magnitude is the Fourier transform over time of the squared spectrogram, $\\mathrm{CS}_x(f,\\alpha)=\\mathcal{F}_{t\\to\\alpha}[|\\mathrm{STFT}_x(f,t)|^2]$, and $\\theta_x$ is the phase of the STFT. The inverse transform $\\mathcal{Z}^{-1}$ rebuilds the STFT from that magnitude by taking the inverse Fourier transform over $\\alpha$ and re-attaching the stored phase, then applies the inverse STFT. CS-SHAP then explains the integrated model $M\\circ\\mathcal{Z}^{-1}$ on CS-domain samples, using SHAP's subset-masking in that domain. The pair lets the paper keep the original end-to-end network intact while changing only the representation in which features are attributed.","core_discovery":"CS-SHAP claims that the right explanation space for intelligent fault diagnosis is the cyclic-spectral domain, because a rotating-machine fault expresses itself as a modulation: a carrier (resonance) frequency $f_c$ and an excitation (modulation) frequency $f_m$. Existing SHAP variants in the time, frequency, time-frequency, or envelope domains expose only one of these two coordinates. The paper's central claim is that attributing in the CS domain, via an integrated model that first maps CS coefficients back to time using the inverse transform, gives Shapley values that simultaneously carry $f_c$ and $f_m$, resolve components that would be conflated in any single-coordinate domain, and remain interpretable under noise.","pith_inferences":["Editorial: The same machinery should transfer to other cyclostationary signals beyond mechanical vibration, such as acoustic emission or motor current signals, where faults also modulate a carrier.","Editorial: A quantitative interpretability metric that compares SHAP salience against known $(f_c, f_m)$ ground truth would make the claimed advantage testable without human inspection.","Editorial: The phase-alignment question in the inverse transform suggests a stress test: sample random CS magnitudes consistent with the background distribution and verify that the reconstructed signals are physically plausible before trusting the resulting attributions."],"forward_implications":["Explanations for a trained fault-diagnosis network can report a single two-dimensional attribution map whose axes are carrier frequency and modulation frequency, instead of one-dimensional frequency or time attributions.","If two fault components share a carrier frequency, or share a modulation frequency, CS-SHAP can still separate their contributions, whereas frequency-only or envelope-only SHAP cannot.","The method stays post-hoc and model-agnostic: any trained network can be wrapped with the inverse transform and explained without retraining or architectural change.","CS-SHAP can be applied to any task whose input is a vibration-like signal with a periodic modulation structure, not just the three datasets demonstrated in the paper."],"supporting_citations":[{"why":"Defines SHAP and the value-function formalism that CS-SHAP reuses for attribution.","marker":"[35]"},{"why":"Supplies the cyclostationary signal theory and the cyclic-spectral representation that motivates the CS domain.","marker":"[34]"},{"why":"Extends SHAP to frequency and time-frequency domains; these are the main comparators that CS-SHAP claims to improve on.","marker":"[32]"},{"why":"Extends SHAP to the envelope domain; this is the comparator that exposes modulation frequency alone.","marker":"[33]"},{"why":"The prior work whose implicit CS computation the authors say they rigorously derive and prove.","marker":"[41]"},{"why":"Uses cyclic spectral coherence with CNNs for bearing fault diagnosis, supporting the use of CS features for fault classification.","marker":"[40]"},{"why":"The SHAP implementation used to compute the Shapley values in the experiments.","marker":"[37]"}],"fun_headline_variants":["SHAP now sees both fault carrier and modulation frequencies","Cyclic-spectral SHAP attributes faults to carrier and modulation","Fault diagnosis explanations now cover carrier and modulation axes","CS-SHAP explains faults by both carrier and modulation frequencies","Dual-frequency SHAP from cyclic-spectral domain clarifies faults"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The inverse CS transform must reconstruct a signal the network can meaningfully score even after SHAP has overwritten parts of the CS magnitude, which in turn assumes the stored phase stays consistent with the sampled magnitudes.","fun_headline_variants_meta":{"raw":{"variants":["SHAP now sees both fault carrier and modulation frequencies","Cyclic-spectral SHAP attributes faults to carrier and modulation","Fault diagnosis explanations now cover carrier and modulation axes","CS-SHAP explains faults by both carrier and modulation frequencies","Dual-frequency SHAP from cyclic-spectral domain clarifies faults"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000858,"raw_usage":{"total_tokens":3731,"prompt_tokens":956,"completion_tokens":2775,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":572,"completion_tokens_details":{"reasoning_tokens":2693}},"tokens_in":572,"tokens_out":2775,"duration_ms":23928,"temperature":1.0,"reasoning_tokens":2693,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T15:27:20.259656+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the outer-race experiment again but with background samples for SHAP drawn from a distribution that assigns random phase per masked CS coefficient, instead of using the phase stored in the sample being explained; if the resulting attribution map changes materially, the explanation depends on an untested phase convention rather than on signal content.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines SHAP and the value-function formalism that CS-SHAP reuses for attribution."},{"cited_title":"Gardner, C","cited_arxiv_id":null,"evidence_quote":"Uses cyclic spectral coherence with CNNs for bearing fault diagnosis, supporting the use of CS features for fault classification."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The SHAP implementation used to compute the Shapley values in the experiments."}],"review_version":1}