{"id":"46330b67-e382-4944-9131-7e145e4351a2","arxiv_id":"2412.01272","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A Bayesian neural network with Bayes-by-backprop is used for gear fault diagnosis, and its uncertainty estimates increase for noisy data and unseen fault classes.","lead":"This paper applies Bayesian neural networks to gear fault diagnosis and shows they can report uncertainty alongside predictions. It demonstrates, on a gearbox testbed with five fault types, that uncertainty rises for noise and for fault classes the model was not trained on.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'alarming uncertainty' claim rests on unvalidated feature discriminability: Section II-B reports no fault-vs-no-fault difference in torque/DC voltage, yet Vdc, Isx, Isy are the BNN inputs and no accuracy or calibration numbers are reported.","rationale":"I read the paper as claiming not just that BNNs are Bayesian, but that their predictive uncertainty can be used as a trustworthy alarm for unseen faults. The necessary condition is that the selected features allow the BNN to separate the seen classes and that high uncertainty is specific to genuinely novel fault conditions. The manuscript provides no quantitative evidence for either. Section II-B is the paper's own warning that torque and DC voltage profiles overlap substantially between fault and no-fault; the subsequent pipeline uses Vdc, Isx, Isy without demonstrating a separability transform. If the model cannot classify seen faults, the 'alarming' uncertainty on eccentricity is confounded by any distribution shift, including load changes. The decomposition in Fig. 7 is also not backed by defined estimators, so the aleatoric/epistemic story is not independently checkable. These are not disagreements with the Bayesian framework; the framework is standard and the qualitative behavior is plausible. They are missing empirical controls. My recommended check, macro-F1/ECE plus OOD-AUROC with non-Bayesian baselines, would settle whether the uncertainty signal is real. I therefore leave the reader's CONDITIONAL verdict unchanged; the paper is promising but under-supported, not demonstrably wrong.","tokens_in":8126,"tokens_out":5693,"duration_ms":54156,"concrete_test":"Reproduce the BNN in Fig. 5(b) on the original three fault classes with inputs Vdc, Isx, Isy; report macro-F1 and expected calibration error for the seen classes. Then score all seen, eccentricity, and high-noise samples by predictive entropy and epistemic uncertainty, and report the AUROC for uncertainty-based OOD detection, comparing against MC Dropout and a deep ensemble. If seen-class macro-F1 is near chance or OOD-AUROC is not clearly above 0.5, the 'alarming uncertainty' claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central operational claim is that BNN uncertainty flags an unseen gear fault, rising to an 'alarming value' in Fig. 6, and that noise raises aleatoric uncertainty in Fig. 7, enabling the trustworthy-diagnosis gate in Fig. 9. For that gate to work, the BNN must first separate the three seen fault classes from the features Vdc, Isx, Isy. Section II-B explicitly says that for the same loading profile, torque and DC-link voltage show no visible statistical difference between a surface fault and no fault, and attributes this to missing filters or poor understanding of parameter interactions. No filtering or feature-engineering step is shown before these signals are fed to the BNN, and no classification accuracy, confusion matrix, or calibration curve is reported anywhere. Without seen-domain accuracy, a high uncertainty on the unseen eccentricity fault cannot be distinguished from a generic load/distribution shift, so it does not by itself validate a trustworthy-diagnosis alarm. In addition, Section III-B only states PU = EU + AU without defining the estimators for the two components, making Fig. 7's 'significant rise in aleatoric uncertainty' non-reproducible. The variational posterior approximation is also undiagnosed, but the feature-discrimination and calibration gap is the more immediate blocker.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes an uncertainty-aware Bayesian neural network (BNN) using Bayes-by-backprop for gear fault diagnosis in a laboratory motor-drive setup. The BNN is trained on signals Vdc, Isx, and Isy for three seen fault classes (No Fault, Missing Tooth, Chipped Tooth) and is then evaluated on an unseen eccentricity fault and on noisy test data. The central claims are that BNN predictive uncertainty rises to an 'alarming value' for the unseen fault (Section IV, Fig. 6), that increased noise produces a significant rise in aleatoric uncertainty (Section IV, Fig. 7), and that incremental retraining with new fault classes reduces diagnostic uncertainty (Section IV, Fig. 8), leading to a trustworthy-diagnosis decision gate (Fig. 9). The paper also provides background on aleatoric versus epistemic uncertainty and on variational inference.","tokens_in":8341,"tokens_out":2764,"duration_ms":25495,"significance":"If the claims were fully supported, the work would make a useful contribution to trustworthy AI for power-electronics fault diagnosis: uncertainty estimates could complement point-estimate accuracy and trigger human intervention for out-of-distribution conditions. The manuscript has strengths worth acknowledging: it uses a real experimental gearbox simulator, compares against CNN and ResNet baselines, and demonstrates an incremental learning workflow. However, the empirical support is mostly qualitative. No classification accuracy, confusion matrix, calibration curve, or repeated-run variability is reported, and the uncertainty decomposition is not defined by equations. The central demonstration is therefore an illustration of known Bayesian properties rather than a validated diagnostic tool.","major_comments":[{"comment":"The input features may not discriminate the fault classes. Section II-B explicitly states that for the same loading profile, torque and DC-link voltage show no visible statistical difference between a surface fault and no fault, and attributes this to missing filters or poor understanding of parameter interactions. Yet the BNN is fed Vdc, Isx, and Isy without any feature-engineering or filtering step being shown, and the paper reports no classification accuracy, confusion matrix, or calibration for the seen-domain classes. Without evidence that the BNN separates the three seen classes from these features, the elevated uncertainty on the unseen eccentricity fault cannot be distinguished from a generic distribution shift, so the central trustworthy-diagnosis claim is unsupported.","section":"Section II-B and Section IV (Fig. 6)"},{"comment":"The uncertainty decomposition is not reproducible as stated. Equation (3) defines predictive uncertainty as the sum of epistemic and aleatoric uncertainty, but no estimators for EU and AU are given in terms of the BNN's predictive distribution or Monte Carlo samples. The claim in Fig. 7 that aleatoric uncertainty rises significantly when SNR is below -25 dB therefore cannot be checked. The authors should provide explicit formulas (for example, based on predictive entropy or mutual information) and state the number of Monte Carlo samples used.","section":"Section III-B, Eq. (3), and Fig. 7"},{"comment":"The paper's own description of Fig. 6 undermines the specificity of the uncertainty alarm. The text says the benchmarking models are 'fairly accurate' for seen faults but also that the BNN's confidence level is 'low for all the three seen fault conditions.' If the BNN gives low confidence even for seen faults, then high uncertainty is not a specific indicator of an unseen fault. A quantitative comparison of predictive confidence and accuracy on seen versus unseen samples, together with a threshold rule for the proposed alarm in Fig. 9, is needed to support the central claim.","section":"Section IV, Fig. 6"},{"comment":"The incremental training results are asserted without quantitative support. The text states that updating the BNN with a new fault class 'automatically minimizes the diagnostic uncertainty and accurately identifies' the fault, but no accuracy or uncertainty numbers are provided for BNN-2, BNN-3, or BNN-4. Because adding the fault class to the training set trivially exposes the model to that class's data, the result is expected; a comparison with retraining from scratch, or at least before/after uncertainty values for the newly added fault, is needed to substantiate the claim.","section":"Section IV, Fig. 8"}],"minor_comments":[{"comment":"Typo: 'high generaltion' should be 'high generalization.'","section":"Section III, first paragraph"},{"comment":"The text says 'see Fig. 5' for the surface-fault versus no-fault comparison, but the relevant figure appears to be Fig. 4; the figure numbering and cross-references should be checked.","section":"Section II-B and Fig. 4/Fig. 5"},{"comment":"The paragraph starting 'We then incorporate the new fault datasets in a step-wise manner...' is repeated almost verbatim for BNN-2, BNN-3, and BNN-4; this should be rewritten or condensed.","section":"Section IV, Fig. 8 captions/description"},{"comment":"The softmax loss in Eq. (2) uses the notation \\hat p without defining it as the softmax output; define \\hat p_d = exp(\\hat y_d)/(sum_{d'} exp(\\hat y_{d'})) before the loss expression.","section":"Section III-A, Eq. (2)"},{"comment":"Reference [14] is a Master's thesis and reference [19] has a typo 'hhttps'; please provide a stable URL or DOI and correct the typo.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper's scope is appropriate for eess.SY and the qualitative story is plausible, but the central operational claim requires quantitative evidence that the BNN is actually accurate on seen classes and that its uncertainty estimates are calibrated. The lack of these numbers, combined with the undefined uncertainty decomposition, makes the current version unsuitable for acceptance. I would encourage the editor to request a revised version with accuracy/calibration results and explicit uncertainty estimators."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nHere is my take on arXiv:2412.01272. The paper applies Bayes-by-backprop to gear fault diagnosis on a physical testbed and shows qualitatively that predictive uncertainty rises for an unseen eccentricity fault and for increased noise. That is real, and the comparison against CNN and ResNet is sensible. The incremental retraining sequence (BNN-1 through BNN-4) is a nice visual demonstration that adding a fault class to training shrinks uncertainty on that class. The authors also deserve credit for acknowledging in Section II-B that the torque and DC-link voltage profiles show no visible statistical difference between surface fault and no-fault; that is a genuine limitation, and admitting it is more than many engineering papers do.\n\nThe soft spots are significant. There is no numeric accuracy, confusion matrix, or calibration metric anywhere. Without seen-domain accuracy, the 'alarming' uncertainty on the eccentricity fault cannot be separated from a generic distribution shift. The paper uses Vdc, Isx, Isy as inputs, yet Section II-B undercuts the discriminative power of those features, and no filtering or feature-engineering step is shown. The uncertainty decomposition in Section III-B is under-specified: PU = EU + AU is stated, but no estimators are given for the two components, so Fig. 7's aleatoric rise is not reproducible. Section IV has a duplicated paragraph in the Fig. 8 descriptions, which suggests a rushed draft. The incremental results are, to a large degree, guaranteed by construction: training on a new fault class lowers uncertainty on that class. That is not a surprise, though it is a useful illustration.\n\nOn the central claim: the qualitative behavior of the BNN follows from standard Bayesian theory, so the paper is not wrong. It is under-validated. There is no code, no data, no comparison to MC Dropout or deep ensembles, and no diagnostic checks on the variational posterior. As an engineering application, this is a plausible first step, but the trustworthy-diagnosis gate in Fig. 9 is not supported by the evidence as presented.\n\nWho is this for? Practitioners in power electronics who want a concrete example of Bayesian uncertainty applied to fault diagnosis will find the testbed and the visualizations useful. Researchers familiar with Bayesian deep learning will see nothing new beyond the application.\n\nI would send it to peer review only under the condition that the authors release code and data and add quantitative metrics: classification accuracy, calibration error, and AUROC for out-of-distribution detection. If that happens, it could be a reasonable conference paper. As is, it is a conditional reject that deserves a serious referee's time because the problem is important and the physical testbed is not something to brush aside.\n\nBest,","headline":"A plausible but under-validated application of Bayes-by-backprop to gear fault diagnosis; the qualitative uncertainty behavior is textbook, but no metrics, code, or data support the 'trustworthy diagnosis' gate.","tokens_in":8894,"tokens_out":2561,"would_cite":false,"duration_ms":21578,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that Bayesian neural networks can quantify prediction uncertainty in gear-fault diagnosis, so a sharp rise in uncertainty flags an unseen fault or noisy data and triggers human intervention.","keywords":["Bayesian neural networks","uncertainty quantification","fault diagnosis","gear faults","motor drives","aleatoric uncertainty","epistemic uncertainty","Bayes by backprop"],"falsifier":"Acquire a held-out set of healthy no-fault samples with the same sensor noise level used in training, and feed them to the trained BNN: if the predictive uncertainty on these in-distribution samples is as high as the uncertainty produced by the unseen eccentricity fault, then the 'alarming uncertainty' signal cannot distinguish seen from unseen conditions. Separately, report the BNN's classification accuracy and expected calibration error on the seen test set; if accuracy is at chance level or the uncertainty is miscalibrated, the claim that rising uncertainty flags untrustworthy diagnosis collapses.","tokens_in":7882,"feed_emoji":"⚙️","tokens_out":6909,"duration_ms":54297,"temperature":0.7,"pith_summary":"This paper sets out to show that Bayesian neural networks (BNNs) can make fault diagnosis in motor drives trustworthy by quantifying the uncertainty in each prediction. The central claim is that a BNN trained on a set of known gear faults will show a sharp rise in predictive uncertainty when it meets a fault class it has never seen, and a comparable rise in aleatoric uncertainty when the input signals become noisy. The paper demonstrates this on an experimental gearbox simulator with three training fault classes, then progressively expands the training set with new fault classes and shows that diagnostic uncertainty drops once a class is learned. If the claim holds, operators get a practical signal for when an AI diagnosis can be trusted and when it should be handed to a human.","feed_headline":"Uncertainty spike flags unseen gear faults in motor drives","feed_subtitle":"Rising uncertainty in Bayesian NN predictions flags new fault types and noisy data for human review.","key_machinery":"The central mechanism is Bayes-by-backprop (BBB) variational inference, in which each network weight is represented by a variational distribution $q_\\theta(\\omega)$ instead of a point estimate. The parameters $\\theta$ are learned by reparameterizing weights as $w = t(\\theta, \\epsilon)$ with a random variable $\\epsilon$, which allows gradients of the variational objective to be computed by backpropagation through the sampled weights (Eqs. 7-8). The trained BNN then outputs a predictive distribution, and the total predictive uncertainty is decomposed into aleatoric uncertainty (data noise) and epistemic uncertainty (model knowledge), per Eq. (3). A deliberately small architecture — two hidden layers of 128 and 64 neurons, one 3-class output layer, stochastic gradient descent at learning rate 0.001 — carries the classification, and the uncertainty decomposition is what the authors use to flag unseen faults and noise.","core_discovery":"The paper's discovery is that Bayesian neural networks, which place probability distributions over network weights, produce a predictive uncertainty that behaves as a reliable alarm for out-of-distribution conditions. Concretely, when the BNN trained on 'No Fault,' 'Missing Tooth,' and 'Chipped Tooth' classes was tested on an unseen 'Eccentricity' fault, the confidence of its class predictions collapsed, with the upper quantile of the predictions dropping to 0.94 under noise, and a 'big discrepancy' appearing for the unseen fault (Fig. 6). When the input signals were corrupted with high noise (SNR below -25 dB), the aleatoric uncertainty rose significantly, dominating the total uncertainty (Fig. 7). The paper also shows an incremental learning property: as new fault labels are added to the training set, the diagnostic uncertainty drops and the newly seen fault is correctly classified (Fig. 8). The proposed use of these findings is a two-stage decision loop: a trustworthy diagnosis when uncertainty is low, and a human-intervention trigger when uncertainty is high (Fig. 9).","pith_inferences":["The paper reports no numeric classification accuracy on the seen test set; a natural next measurement is to compare BNN accuracy with the CNN and ResNet baselines on the same folds, to see whether the uncertainty benefit costs accuracy.","The 'alarming value' of uncertainty is described qualitatively; converting the flag into a calibrated threshold (e.g., via reliability diagrams) would be the deployment step that turns Fig. 9 into a working alarm.","Since Section II-B shows the torque and DC-voltage profiles overlap between surface-fault and no-fault cases, an informative extension is to train the BNN on each electrical signal separately (Vdc, Isx, Isy) to isolate which feature actually carries the discriminatory information that the uncertainty signal depends on.","The variational posterior's accuracy is assumed rather than checked; comparing BBB uncertainty with a gold-standard MCMC posterior on a small subset would test whether the reported epistemic uncertainty is trustworthy."],"forward_implications":["A BNN can serve as a gatekeeper: when predictive uncertainty exceeds a threshold, the diagnosis is flagged untrustworthy and referred to a human, rather than being accepted on the strength of a single point-estimate probability.","Adding new fault classes to the training set reduces diagnostic uncertainty and corrects prior misclassifications, which means the same model can be updated incrementally as new fault data becomes available.","Because noise raises aleatoric uncertainty while missing data raises epistemic uncertainty, the decomposition gives operators a diagnosis of why the model is unsure, not just that it is unsure.","The approach extends beyond gears to other power-electronics components where fault data is scarce and where overconfident deep-learning outputs are a known risk."],"supporting_citations":[{"why":"Supplies the Bayes-by-backprop algorithm that the paper uses to train the BNN's variational posterior.","marker":"[18]"},{"why":"Prior probabilistic Bayesian deep learning framework for machine fault diagnosis that this work extends to motor-drive gear faults.","marker":"[13]"},{"why":"Tutorial source for the variational inference formulation used to approximate the posterior p(ω|X,Y).","marker":"[17]"},{"why":"Master's thesis that provides the SpectraQuest gearbox simulator setup and the fault dataset used for training and testing.","marker":"[14]"}],"fun_headline_variants":["Bayesian NN uncertainty alarms unseen gear faults","Uncertainty as a tripwire for novel gear faults","Rising model uncertainty pinpoints unseen gear faults","Confidence drop reveals new motor drive faults","AI doubt flags fresh fault types in motor drives"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the three electrical signals (DC-link voltage and the two stator currents) actually discriminate the gear faults, since the paper reports overlapping torque and DC-voltage profiles for fault vs. no-fault and gives no numeric accuracy on the seen test set.","fun_headline_variants_meta":{"raw":{"variants":["Bayesian NN uncertainty alarms unseen gear faults","Uncertainty as a tripwire for novel gear faults","Rising model uncertainty pinpoints unseen gear faults","Confidence drop reveals new motor drive faults","AI doubt flags fresh fault types in motor drives"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000163,"raw_usage":{"total_tokens":1230,"prompt_tokens":919,"completion_tokens":311,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":240}},"tokens_in":535,"tokens_out":311,"duration_ms":3094,"temperature":1.0,"reasoning_tokens":240,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:30:19.483122+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Acquire a held-out set of healthy no-fault samples with the same sensor noise level used in training, and feed them to the trained BNN: if the predictive uncertainty on these in-distribution samples is as high as the uncertainty produced by the unseen eccentricity fault, then the 'alarming uncertainty' signal cannot distinguish seen from unseen conditions. Separately, report the BNN's classification accuracy and expected calibration error on the seen test set; if accuracy is at chance level or the uncertainty is miscalibrated, the claim that rising uncertainty flags untrustworthy diagnosis collapses.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior probabilistic Bayesian deep learning framework for machine fault diagnosis that this work extends to motor-drive gear faults."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Tutorial source for the variational inference formulation used to approximate the posterior p(ω|X,Y)."},{"cited_title":"Biswas, ”Intelligent motor fault detection”, Master’s thesis, Univer- sity of South Denmark, 2023","cited_arxiv_id":null,"evidence_quote":"Master's thesis that provides the SpectraQuest gearbox simulator setup and the fault dataset used for training and testing."}],"review_version":1}