{"id":"c80fc6e5-fd86-4ee0-9d6b-2e95bc255139","arxiv_id":"2501.04819","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Skip-connection and transformer-enhanced convolutional autoencoders detect expert-labeled anomalies in real wood planer audio with AUC up to 0.875, on a newly released factory dataset.","lead":"A new open dataset of real wood planer sounds is used to train autoencoders that flag abnormal events like stuck or broken boards, reaching an AUC of 0.875. The result is a practical step toward low-cost acoustic monitoring for sawmills, but the method is an incremental variant of existing deep anomaly detection architectures.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported AUC and model ranking are computed against an evaluation set the authors admit is incompletely labeled; unlabeled anomalies scored as normal could distort both AUC and which model wins.","rationale":"The central claim is empirical: Skip-CAE-Transformer achieves AUC 0.875 and beats baselines on this new dataset. The truth of that claim requires the evaluation labels to be a trustworthy ground truth. The paper itself concedes they are not guaranteed complete (Section III). This is not a matter of disagreement with consensus; it is an internal admission of missing support. Without an estimate of the unlabeled-anomaly rate or a sensitivity analysis, the reported AUC could be biased and the ranking could shuffle. The per-type analysis is even more fragile because 'broken board' AUC is computed from only 4 positive examples. The reader's weakest assumption correctly identifies this. I considered whether the absence of error bars and seed repetitions is more load-bearing; it is real, but the label incompleteness is the more fundamental threat because even a perfectly repeated experiment would still measure the wrong target. The proposed re-annotation test is feasible and would settle whether the concern lands. The paper promises code and data on GitHub, which makes the test practical. Since the verdict CONDITIONAL already conditions on addressing exactly this kind of issue, no change to the reader's verdict is needed.","tokens_in":7691,"tokens_out":6296,"duration_ms":65105,"concrete_test":"Have a second expert independently re-annotate all 3,235 evaluation recordings (or a stratified random sample of at least 500) for anomalies using the same definitions, then recompute the overall and per-type AUC/pAUC for all models using the union of both experts' labels. If the Skip-CAE-Transformer is still top-ranked and the AUC gap to Skip-CAE remains outside a bootstrap 95% CI, the concern is mitigated; otherwise the headline result is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III states: 'since the recordings come from real-life operations, some anomalies might not be labeled in the evaluation set.' The evaluation AUC treats every unlabeled 10-second recording as normal. There are 3,235 evaluation recordings but only 105 labeled anomaly events; because the recordings contain the sound of boards being planed, unlabeled stuck or uneven-board events may be present. A model that correctly assigns high anomaly scores to those unlabeled events is charged a false positive, so the reported AUCs (0.875, 0.846, 0.798) are not true AUCs and the ranking between models could change: a detector with better sensitivity to the unlabeled anomalies is penalized more. The problem is compounded by the per-type table: 'broken board' results rest on only 4 positive events, making those AUC values highly unstable. No confidence intervals or seed repetitions are provided, so there is no way to tell whether the observed gaps reflect real superiority or label noise.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper addresses acoustic anomaly detection for industrial wood planers. The authors introduce a new dataset of 7,562 ten-second recordings from a real planing mill, with 105 expert-labeled anomaly events, and propose two neural architectures: a convolutional autoencoder with skip connections (Skip-CAE) and a variant with a transformer encoder-decoder (Skip-CAE-Transformer). The models are compared against a DCASE baseline autoencoder, the convolutional autoencoder of Duman et al., one-class SVM, and isolation forest, using AUC and pAUC on a held-out evaluation set. The reported best AUC is 0.875 for the Skip-CAE-Transformer.","tokens_in":7876,"tokens_out":4245,"duration_ms":40496,"significance":"If the reported results are robust, the paper contributes a useful open industrial dataset and demonstrates that skip connections and attention can improve reconstruction-based acoustic anomaly detection. The public dataset and code are a clear strength. However, the central comparative claim is currently supported only by point estimates on a single evaluation set with acknowledged incomplete labels, and the paper lacks uncertainty quantification, so the practical significance of the contribution depends on addressing these limitations.","major_comments":[{"comment":"The evaluation-set AUCs are computed under the assumption that every unlabeled ten-second recording is normal, yet Section III states that \"some anomalies might not be labeled in the evaluation set.\" Because unlabeled anomalous recordings would be counted as false positives when a detector assigns them high anomaly scores, the reported AUC values (e.g., 0.875 for Skip-CAE-Transformer) are not verified against complete ground truth, and the ranking of models could change if the better detector is also better at catching the unlabeled events. The authors should re-verify the evaluation-set labels with the expert, quantify the expected number of missing labels, or provide a sensitivity analysis showing that the ranking is stable under plausible label noise.","section":"Section III and Section V-A"},{"comment":"The paper reports single point estimates without confidence intervals, repeated training runs, or significance tests. The margin between Skip-CAE (AUC 0.846) and Skip-CAE-Transformer (0.875) is only 0.029, and the per-type results in Table II are based on very few positive events (four broken boards); without error bars or bootstrap intervals, the statement that \"the Skip-CAE performs significantly better\" (Section V-A) is not statistically justified. Add confidence intervals (e.g., bootstrap over evaluation recordings) and at least a few training seeds, and restrict per-type conclusions to anomaly types with sufficient positive samples.","section":"Section V-A and Table II"},{"comment":"The claim that \"Skip-CAE-Transformer outperforms all other models in AUC or pAUC for all of the anomaly types\" is contradicted by Table II: for Broken Board, Skip-CAE has AUC 0.777 versus 0.743, and for Uneven or Thick Wood, Skip-CAE has pAUC 0.820 versus 0.807. Please correct the claim or qualify it to \"best in at least one of the two metrics for most types\" and discuss where it is not the best.","section":"Section V-B and Table II"},{"comment":"The statement that Skip-CAE-Transformer \"performs the best\" is only true for AUC (0.875 versus 0.846); on pAUC, Skip-CAE reports 0.787 versus 0.785, so the overall superiority depends on the metric. The text should state which metric is being prioritized and discuss the pAUC comparison explicitly.","section":"Section V-A and Figure 3"}],"minor_comments":[{"comment":"There is a typo in \"convolutional autocencoders\" in the first sentence of Section IV; it should read \"autoencoders.\"","section":"Section IV"},{"comment":"The DCASE baseline is described as an autoencoder whose input and output \"lead to 32,080 neurons\"; please clarify that this is the flattened mel-spectrogram dimension and state whether any dropout or other regularization is applied.","section":"Section IV"},{"comment":"The ROC curves in Figure 3 are difficult to distinguish in grayscale; consider using different line styles or markers in addition to color.","section":"Figure 3"},{"comment":"The contribution bullet describing the dataset should state the license and any conditions of use, since the dataset is a central contribution.","section":"Section I"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of an applied machine-learning-for-audio venue, and the dataset release is valuable. The main technical issue is that the evaluation labels are acknowledged to be incomplete and the reported performance differences lack uncertainty quantification; these are fixable with additional analysis rather than representing a fundamental flaw in the approach."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The real contribution here is the dataset: 21 hours of real planer sounds with 105 expert-labeled anomalies across three types, plus promised code and data. That is genuinely new and likely to be reused by anyone working on industrial acoustic monitoring. The paper itself is a straightforward application of established ideas — skip connections and transformer blocks on convolutional autoencoders — and the authors do not oversell the architecture as a new principle. The experimental setup is clean: a single held-out evaluation set, AUC and pAUC metrics, and comparisons against four sensible baselines including the Duman et al. CAE. Reported improvements are plausible.\n\nThe soft spots are real but addressable. The authors admit that some anomalies in the evaluation set may be unlabeled; if so, the AUC treats those recordings as normal, which can bias the reported numbers and even change the model ranking. That is not a fatal flaw, but it needs to be quantified — ideally by re-labeling a sample or using a partial-label evaluation. More importantly, there are no error bars, no repeated seeds, and no significance tests. The phrase 'significantly better' is used without statistical support. The per-type table compounds this: the 'broken board' AUC rests on four positive events, so those numbers are unstable. These are standard revision requests, not fundamental problems.\n\nOne small thing: the DCASE baseline is modified to fit the dataset, and the Skip-CAE-Transformer's pAUC (0.785) is slightly below Skip-CAE's (0.787), though the text focuses on AUC. That is worth mentioning but does not change the overall picture.\n\nI think this deserves a serious referee. The dataset and the application are valuable to the anomaly detection and wood-processing communities, and the methods are clearly described. The evaluation needs strengthening before publication, but the work is honest and reproducible. Send it to review, with the expectation that the authors add uncertainty quantification and address the label incompleteness.","headline":"A useful new dataset and a solid but statistically unquantified application of known autoencoder variants to wood planer acoustic anomaly detection.","tokens_in":585,"tokens_out":819,"would_cite":false,"duration_ms":23299,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two neural nets spot wood planer faults by ear, top AUC 0.875","keywords":["acoustic anomaly detection","convolutional autoencoder","skip connections","transformer","wood planer","industrial monitoring","mel spectrogram","unsupervised learning"],"falsifier":"Re-annotating the evaluation set with exhaustive labels and recomputing the AUC would settle the claim: if many currently unlabeled recordings are anomalies, the reported 0.875 AUC and the model ranking could change.","tokens_in":7542,"feed_emoji":"🎧","tokens_out":3982,"duration_ms":33782,"temperature":0.7,"pith_summary":"The paper aims to show that deep convolutional autoencoders can detect abnormal sounds in industrial wood planers using only the sound of normal operation for training. On a newly collected real-factory dataset of 7,562 ten-second recordings, the proposed Skip-CAE and Skip-CAE-Transformer reach areas under the ROC curve of 0.846 and 0.875, outperforming a published convolutional autoencoder, the DCASE autoencoder baseline, one-class SVM, and isolation forest. The motivation is practical: skilled planer operators are scarce, and acoustic monitoring could give novice operators an automated ear for stuck boards, broken boards, and uneven or thick wood passing through the machine.","feed_headline":"Two neural nets spot wood planer faults by ear, top AUC 0.875","feed_subtitle":"Skip-connection autoencoders beat classical baselines on a new real-factory audio dataset of 105 labeled anomalies.","key_machinery":"The core mechanism is the reconstruction-error anomaly score: an autoencoder is trained to reconstruct normal mel spectrograms, and a high reconstruction error marks an anomaly. The Skip-CAE is a convolutional autoencoder with skip connections between encoder and decoder layers, using leaky ReLU and batch normalization before pooling and upsampling to stabilize training and preserve fine spectral detail. The Skip-CAE-Transformer inserts a transformer encoder-decoder at the bottleneck (10 heads, single layer) and routes two skip connections through fully connected layers, which lets the network decide how much detail to pass; this transformer attention is what the paper credits for the further improvement in anomaly detection.","core_discovery":"The central claim is that adding skip connections to a convolutional autoencoder, and further adding transformer encoder/decoder blocks, yields a practical acoustic anomaly detector for real sawmill noise. Trained only on normal planer sounds (recordings made shortly after knife jointing or head change), the models flag anomalies by reconstruction error on log-scaled mel spectrograms. On the evaluation set spanning two days of operation with 105 expert-flagged anomalies, Skip-CAE-Transformer achieves the best overall AUC of 0.875 (pAUC 0.785), and Skip-CAE achieves 0.846 (pAUC 0.787); both are the only models reaching a 20% true positive rate with zero false positives. Per anomaly type, Skip-CAE-Transformer leads in AUC or pAUC for all three categories, with its largest margin on uneven or thick wood (AUC 0.921).","pith_inferences":["Editorial inference: the method's dependence on the expert label set is the main risk; if the evaluation set contains unlabeled anomalies, the reported AUC is an upper bound, and the ranking between models could shift under complete labeling.","Editorial inference: the same Skip-CAE-Transformer recipe could plausibly transfer to other rotating wood-processing machinery, such as saws or CNC routers, but this would require re-tuning and is not demonstrated here.","Editorial inference: a practical extension would be to test whether the models can localize anomalies in time within a 10-second clip, which the current clip-level AUC does not address."],"forward_implications":["If the results hold, acoustic monitoring with these models could be deployed on real planers to alert operators to stuck boards, broken boards, and uneven or thick wood in near real time.","The public dataset of 7,562 real-factory recordings gives other researchers a benchmark for industrial acoustic anomaly detection under realistic noise.","The finding that skip connections and a transformer bottleneck improve over the DCASE baseline supports further work on attention-based autoencoders for machine-condition monitoring.","The models' ability to reach 20% true positive rate with no false positives suggests they can support high-precision alarm systems in settings where false alarms are costly.","Because training uses only normal sounds, the approach fits unsupervised deployment scenarios where anomaly examples are rare or unknown."],"supporting_citations":[{"why":"Supplies the convolutional autoencoder architecture that Skip-CAE and Skip-CAE-Transformer modify, and serves as a published baseline to beat.","marker":"[2]"},{"why":"Defines the DCASE 2024 challenge autoencoder baseline and the AUC/pAUC evaluation protocol used in the experiments.","marker":"[19]"},{"why":"Provides the deep feedforward autoencoder architecture used as the DCASE baseline in this paper.","marker":"[21]"},{"why":"Supplies the transformer encoder-decoder implementation added to Skip-CAE-Transformer.","marker":"[22]"},{"why":"Provides the scikit-learn implementations of one-class SVM and isolation forest used as classical baselines.","marker":"[25]"}],"fun_headline_variants":["Wood planer faults caught by sound: AI hits 0.875 AUC","Deep learning ears for wood planers: best AUC 0.875","Skip-connection autoencoders listen for planer glitches","Acoustic AI for sawmills: 0.875 AUC on real factory noise","Transformers + autoencoder improve planer fault detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The expert anomaly labels in the evaluation set are assumed complete enough to compute a trustworthy AUC, but the paper states some anomalies may be unlabeled, so unlabeled anomalous recordings are scored as normal.","fun_headline_variants_meta":{"raw":{"variants":["Wood planer faults caught by sound: AI hits 0.875 AUC","Deep learning ears for wood planers: best AUC 0.875","Skip-connection autoencoders listen for planer glitches","Acoustic AI for sawmills: 0.875 AUC on real factory noise","Transformers + autoencoder improve planer fault detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000714,"raw_usage":{"total_tokens":3208,"prompt_tokens":941,"completion_tokens":2267,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":557,"completion_tokens_details":{"reasoning_tokens":2173}},"tokens_in":557,"tokens_out":2267,"duration_ms":15496,"temperature":1.0,"reasoning_tokens":2173,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:24:18.258295+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotating the evaluation set with exhaustive labels and recomputing the AUC would settle the claim: if many currently unlabeled recordings are anomalies, the reported 0.875 AUC and the model ranking could change.","supporting_citations":[{"cited_title":"Acoustic anomaly detection using convolutional autoencoders in industrial processes,","cited_arxiv_id":null,"evidence_quote":"Supplies the convolutional autoencoder architecture that Skip-CAE and Skip-CAE-Transformer modify, and serves as a published baseline to beat."},{"cited_title":"Attention is All you Need,","cited_arxiv_id":null,"evidence_quote":"Supplies the transformer encoder-decoder implementation added to Skip-CAE-Transformer."}],"review_version":1}