{"id":"5bfa3999-3592-4cf9-996d-15588b16c25e","arxiv_id":"2601.08480","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Better proxy-task performance does not generally improve anomalous sound detection; only source separation showed a strong, consistent positive correlation.","lead":"This paper tests whether doing better at self-supervised 'proxy' tasks (reconstructing sounds, classifying machine IDs, separating noise, contrastive learning, pretrained models) makes anomalous sound detection better. It finds the link often breaks, with source separation the only clear exception, and proposes a three-step design checklist.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Source-separation correlation (Table IV) is confounded with architecture: more conformer blocks/channels increase both SI-SDRi and the feature dimension concatenated for ASD, so 'separation quality improves ASD' is not isolated from capacity.","rationale":"The paper's broad negative finding—that proxy-task metric improvements do not generally translate to ASD gains—is reasonably supported by the saturated classification F1-scores, the collapsed contrastive training, and the inconsistent pre-trained-model rankings. I do not dispute that part. But the paper's distinctive positive claim, and the motivation for its three-stage protocol, rests on the source-separation correlation. That is the least secure component because the experimental design does not manipulate separation quality independently of architecture: varying N and channel width changes both the SI-SDRi and the size/dimensionality of the concatenated ASD features. Thus the strong Spearman rho may be an artifact of capacity or feature richness rather than evidence that improved separation quality per se improves anomaly detection. This matches the reader's weakest assumption, and I add the specific observation that the ASD feature representation itself changes dimensionality across the same configurations. The LP protocol's use of anomalous labels is a real but secondary weakness, affecting all methods rather than the unique positive claim. Because the reader already assigned CONDITIONAL with this concern, my independent read does not move the verdict; the paper needs capacity-controlled experiments and repeated runs before the source-separation correlation can be accepted as causal.","tokens_in":18820,"tokens_out":4569,"duration_ms":49050,"concrete_test":"Fix the architecture and feature extraction to the best separation configuration (4-block, 128-ch). Train this same model with multiple random seeds and early-stopping checkpoints (or by varying the training mixture SNR range) to obtain a spread of SI-SDRi values spanning roughly 2–4 dB. For each checkpoint, compute in-domain LP, out-domain LP, and MD AUC. If ASD metrics track SI-SDRi monotonically within a fixed architecture and fixed feature dimensionality, the correlation claim is supported; if ASD metrics stay flat or vary independently, the Table IV correlation is a capacity/feature-dimension artifact. Report at least 3 seeds per checkpoint to provide error bars.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's only positive result—that source separation shows a strong, consistent correlation with ASD—is built on configurations in Sec. III-D that vary the number of conformer blocks N (0,1,2,4) and channel width (64,128). Two things change together: the proxy metric SI-SDRi, and the ASD feature representation, which concatenates channel-pooled features from N+1 extraction points and therefore grows in dimensionality with N and width. Table IV thus cannot distinguish 'better separation causes better ASD' from 'a larger/richer feature extractor yields better ASD.' The reported Spearman coefficients (ρ ≈ 0.95–0.98, p<0.01) are computed over only 8 points, all arranged along this capacity-ordered grid. A third variable—architecture size, feature dimension, parameter count—can fully explain the monotonic ordering. Even within fixed width, the trend is not perfectly monotonic (e.g., 2-block/128-ch MD=59.24 vs 1-block/128-ch MD=60.51), suggesting SI-SDRi alone is not the driving factor. Since this correlation is the basis for the paper's headline claim and the proposed Stage-3 verification protocol, the capacity/feature-dimension confound is load-bearing. A secondary issue is that the LP 'ASD performance' is obtained by training a linear classifier with anomalous labels from the evaluation set (Sec. IV-B1), so it is not fully unsupervised; however, the structural confound is the primary concern.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates whether optimizing proxy-task metrics (AE reconstruction error, classification accuracy, SI-SDRi, contrastive alignment/uniformity, and AudioSet mAP) transfers to anomalous sound detection (ASD) performance measured by linear-probe AUC and Mahalanobis-distance AUC on ToyADMOS2/MIMII under the DCASE 2022 setup. Across five proxy-task families and 41 configurations, it finds that strong proxy performance does not generally improve ASD: classification saturates, contrastive learning collapses or underperforms, and only source separation shows a strong positive Spearman correlation. The paper additionally proposes a three-stage alignment verification protocol for designing proxy tasks.","tokens_in":19230,"tokens_out":7039,"duration_ms":72311,"significance":"If the source-separation result were causal, the paper would be a valuable contribution: it would systematically challenge the common assumption that better proxy performance implies better ASD, and its three-stage protocol would give practitioners a concrete design checklist. The study is broad in scope and thoughtfully uses both a supervised linear probe and an unsupervised Mahalanobis score, with exact Spearman p-values. It also offers a useful taxonomy of failure modes (saturation, collapse, task misalignment). The main limitation is that the sole positive correlation is confounded with model capacity and the ASD feature-vector dimensionality, so the paper's headline claim is not yet established.","major_comments":[{"comment":"The headline positive result—that source separation is the only task with a strong positive proxy–ASD correlation—is confounded. The eight configurations vary the number of conformer blocks N ∈ {0,1,2,4} and channel width ∈ {64,128}. Two things change simultaneously: model capacity, and the ASD feature representation, which concatenates channel-pooled features from N+1 extraction points (Sec. III-D), so the ASD feature dimensionality grows with N. The reported Spearman coefficients (ρ ≈ 0.95–0.98, n=8) therefore cannot distinguish 'better separation causes better ASD' from 'a larger/richer feature extractor yields better ASD.' In fact, Table IV is not even consistently monotonic: 2-block/128-ch has higher SI-SDRi (3.57 dB) than 1-block/128-ch (3.40 dB) but lower MD AUC (59.24% vs 60.51%). To support the causal-sounding claim, the paper needs to control for capacity/feature dimension—for","section":"Section III-D, Table IV"},{"comment":"The LP protocol trains a linear classifier on the evaluation set with access to both normal and anomalous labels. Thus the 'ASD performance' reported for LP is not an unsupervised detector; it is a supervised representation-quality probe. The abstract and conclusions state the findings in terms of 'anomalous sound detection performance' without this caveat. This matters because the source-separation positive correlation is reported for LP as well as MD. The paper should either consistently describe LP as a supervised measure of linear separability and restrict 'unsupervised ASD' claims to the MD results, or provide evidence that LP correlates with a fully unsupervised scoring on the same features.","section":"Section IV-B1, Eqs. (1)-(4)"},{"comment":"The quantitative conclusions are based on single runs with no seeds or error bars. All AUC, F1, and SI-SDRi values are point estimates, and the Spearman correlations are computed over only n=4–10 configurations. For example, the AE in-domain LP AUC differences of about 1% (Table II) are interpreted as trends, but run-to-run variability could be of the same magnitude. The exact p-values do not account for this. Reporting multiple random seeds with mean and standard deviation, or at least a sensitivity analysis, would substantially strengthen the reliability of the correlation findings.","section":"Tables II-VII"}],"minor_comments":[{"comment":"For contrastive learning, it is unclear whether the reported Spearman ρ uses alignment or uniformity as the proxy metric. The footnote explains the sign convention for 'low is better' metrics, but the row labels 'Contrastive (SimCLR)' and 'Contrastive (SimSiam)' do not state which metric was used.","section":"Table VII"},{"comment":"The captions refer to 'normalized proxy task performance,' but no normalization formula is given. Please specify the normalization (e.g., min-max across configurations) and whether it is applied per proxy task or globally.","section":"Figs. 2-4"},{"comment":"The column header 'Mahalanobis(↑)' is ambiguous: it is later described as AUC, but the header does not say so. Please make the metric explicit in each table.","section":"Tables II and IV"},{"comment":"The architecture is cited to the authors' previous work [27]. For reproducibility, briefly state which hyperparameters (conformer blocks, channel widths, training budget) are newly varied here and whether the remaining settings are identical to [27].","section":"Section III-D"},{"comment":"The claim 'consistent positive correlation across all evaluation metrics (ρ > 0.95, p < 0.01)' should be qualified by the small n and the capacity confound discussed above; currently the sentence appears before the confound is acknowledged.","section":"Section VI-A"}],"recommendation":"major_revision","confidential_remarks":"The central problem is the capacity/feature-dimension confound in the source-separation experiment. If the authors can provide a controlled analysis that separates separation quality from feature-extractor capacity, the paper's positive claim can be made credible. Without that, the headline result is not supported, though the negative results (AE, classification, contrastive, pre-trained) remain suggestive. I recommend major revision rather than rejection because the confound can in principle be addressed with additional experiments within the paper's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The negative findings here are probably right, and the paper is worth reading for them. The positive finding—that source separation quality strongly correlates with ASD performance—is not yet established. The correlation in Table IV is built on configurations that vary the number of conformer blocks and channel widths, which changes both the SI-SDRi and the feature vector fed to the anomaly scorer (concatenated channel-pooled features from N+1 extraction points). So better separation and a larger/richer feature extractor move together; the Spearman rho ~0.95 over only 8 points does not isolate separation quality. There is even a non-monotonic tick within fixed width (1-block/128-ch MD 60.51 vs 2-block/128-ch 59.24), which undercuts the monotonic story.\n\nWhat is good: this is the first systematic ASD-domain comparison of proxy-task families (AE, classification, source separation, contrastive, pretrained) on ToyADMOS2/MIMII with two evaluation protocols. The saturated classification task, the contrastive collapse, and the failure of AudioSet mAP to predict ASD transfer are useful, honestly reported observations. The paper is transparent about showing all configurations, and the proposed three-stage verification protocol is a reasonable design checklist even if it sounds like common sense after the fact.\n\nOther soft spots: the LP protocol trains a linear classifier using anomalous labels from the evaluation set, so it is a supervised separability probe, not fully unsupervised ASD. The split description is inconsistent (300 samples per section versus 200 per section from the earlier dataset description). No seeds or error bars, no code/data release, and per-task correlations are computed on 4-10 points, so the p-values should be read with a grain of salt.\n\nThe broad negative result—strong proxy performance does not guarantee better ASD—is robust across tasks and unlikely to be overturned. The source-separation claim needs capacity-controlled experiments (e.g., fix feature dimension, vary separation quality through training signal; or vary capacity while holding feature dimension constant), repeated runs, and artifact release before it can support the paper's headline.\n\nWho it is for: anyone working on ASD proxy tasks. Worth a serious referee; I would send it out, but expect major revision. I would not cite the positive result as it stands, but the negative map is citable once the paper is cleaned up.","headline":"Useful negative map of proxy-task/ASD correlations, but the only positive result (source separation) is confounded with model capacity and needs controlled experiments.","tokens_in":19637,"tokens_out":3367,"would_cite":true,"duration_ms":32755,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Source separation is the only proxy task whose improvements consistently improve anomalous sound detection.","keywords":["anomalous sound detection","proxy tasks","self-supervised learning","source separation","linear probe","Mahalanobis distance","task alignment","representation learning"],"falsifier":"Train separation models with identical architecture and capacity but differing separation quality—for example, by varying the training SNR range or adding a reconstruction auxiliary loss—and check whether ASD AUC follows SI-SDRi; a flat or non-monotonic ASD curve across SI-SDRi values would refute the paper's central correlation claim.","tokens_in":1262,"feed_emoji":"","tokens_out":5782,"duration_ms":69486,"temperature":0.7,"pith_summary":"The paper asks whether improving a self-supervised proxy task—autoencoding, classification, source separation, contrastive learning, or pretrained feature extraction—actually improves anomalous sound detection (ASD). Across dozens of configurations on two public machine-sound datasets, it measures each task's own metric and correlates it with ASD performance measured by linear probes and Mahalanobis distance. The central finding is that the usual assumption does not hold in general: only source separation shows a strong, consistent positive correlation, while classification saturates at near-perfect accuracy without helping detection, contrastive learning collapses as model size grows, autoencoders correlate only with distributional compactness, and pretrained model accuracy does not transfer. If true, proxy metrics cannot be trusted as surrogates for ASD performance, and designing proxy tasks requires checking both task difficulty and objective alignment.","feed_headline":"Source separation alone predicts anomaly-detection gains","feed_subtitle":"Other self-supervised tasks plateau or collapse; a 3-stage check realigns proxy design with detection.","key_machinery":"The central instrument is the correlation analysis between proxy-task metrics and ASD metrics across controlled capacity variations. Proxy metrics are the AE reconstruction MAE, classification Macro-F1, source-separation SI-SDRi, contrastive alignment/uniformity, and pretrained AudioSet mAP. ASD performance is scored by a linear probe (trained on evaluation-set anomalous labels) and by Mahalanobis distance on normal training features, both summarized as AUC. Spearman rank correlation across configurations isolates whether proxy improvements track detection gains; the source-separation experiments vary conformer-block count and channel width, producing a monotone proxy-ASD relationship that t","core_discovery":"The paper claims that improved proxy task performance does not necessarily improve anomalous sound detection performance. It reports that source separation is the only task among the five studied whose proxy metric (SI-SDRi) strongly and consistently correlates with ASD performance across all evaluation protocols (Spearman rho above 0.95, p < 0.01), such that improved separation quality translates into better detection. Classification tasks saturate at F1-scores above 97%, losing discriminative power; contrastive learning fails to learn meaningful features due to limited data diversity and collapses as capacity grows; autoencoders correlate only with Mahalanobis-distance compactness, not lin","pith_inferences":["The source-separation correlation is measured by scaling model capacity, so the claim that 'improved separation improves detection' may partly reflect a capacity effect; a test that varies separation quality at fixed capacity would clarify causality.","The linear-probe evaluation trains on anomalous labels from the evaluation set, so the reported LP scores measure a semi-supervised separability rather than pure unsupervised ASD; Mahalanobis distance is the closer approximation to the true unsupervised setting.","The separation model's own reconstruction residual could be used directly as an anomaly score, potentially making the proxy and target objectives the same; this extension is not tested in the paper.","The 'task difficulty' explanation predicts an inverted-U relationship: tasks too easy saturate while tasks too hard collapse; adding a curriculum or SNR control could test whether a middle difficulty yields the strongest proxy-ASD alignment."],"forward_implications":["Proxy-task metrics should not be used alone as surrogates for ASD performance; a task like classification can reach ceiling accuracy while yielding poor detection features.","For reconstruction-based ASD, the common assumption that better reconstruction implies better detection is only partially supported: reconstruction quality tracks distributional compactness (Mahalanobis distance) but not linear separability.","Source separation stands out as a proxy whose optimization directly benefits anomaly detection, suggesting separation-style objectives are a safer design choice for ASD systems.","Contrastive learning on small, low-diversity acoustic datasets is prone to feature collapse, so capacity scaling without data diversity can degrade ASD performance.","The proposed three-stage protocol—checking proxy-task health, evaluating representation suitability, and running correlation analysis across hyperparameters—provides a concrete diagnostic for future ASD proxy-task design."],"fun_headline_variants":["Only source separation boosts anomaly detection","Proxy task strength doesn't ensure better anomaly detection","ASD proxy gains don't translate to detection performance","Source separation alone predicts sound anomaly detection","Why contrastive and classification proxy tasks fail for ASD"],"cache_read_input_tokens":20992,"weakest_assumption_plain":"The source-separation correlation is measured by scaling model size, so if larger capacity alone drives both separation and detection improvements, the claim that improved separation quality itself drives ASD gains would collapse.","fun_headline_variants_meta":{"raw":{"variants":["Only source separation boosts anomaly detection","Proxy task strength doesn't ensure better anomaly detection","ASD proxy gains don't translate to detection performance","Source separation alone predicts sound anomaly detection","Why contrastive and classification proxy tasks fail for ASD"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000221,"raw_usage":{"total_tokens":1289,"prompt_tokens":746,"completion_tokens":543,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":490,"completion_tokens_details":{"reasoning_tokens":475}},"tokens_in":490,"tokens_out":543,"duration_ms":5456,"temperature":1.0,"reasoning_tokens":475,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T10:48:58.914518+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train separation models with identical architecture and capacity but differing separation quality—for example, by varying the training SNR range or adding a reconstruction auxiliary loss—and check whether ASD AUC follows SI-SDRi; a flat or non-monotonic ASD curve across SI-SDRi values would refute the paper's central correlation claim.","supporting_citations":[],"review_version":1}