Pith. sign in

REVIEW 4 major objections 5 minor 34 references

AquaSignal: An Integrated Framework for Robust Underwater Acoustic Analysis

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AquaSignal is a modular pipeline that denoises underwater recordings with a U-Net, classifies known ship types with a ResNet18, and flags never-seen acoustic events with an autoencoder, reporting 71.3% classification accuracy and 91.5%…

desk verdict The paper's own numbers contradict its central claim—denoising hurts classification and novelty detection is below a trivial baseline—but the integration and honest discussion of the trade-off give it enough substance to referee, not enough to accept as-is. read the letter →

arxiv 2505.14285 v1 pith:GGPWSSNT submitted 2025-05-20 cs.SD cs.AIeess.AS

Eirini Panteli , Paulo E. Santos , Nabil Humphrey This is my paper · ORCID
classification cs.SDcs.AIeess.AS
keywords underwateracousticssignaldenoisingshipclassificationnoveltydetectionautoencoderResNet18U-NetDeepdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AquaSignal asks whether underwater acoustic monitoring can be handled by one integrated system rather than by separate tools bolted together. The pipeline turns raw hydrophone recordings into two-second clips, turns each clip into a spectrogram (a time-frequency picture of the sound), cleans it with a U-Net denoiser, sorts known ship types with a ResNet18 classifier, and flags anything unfamiliar with an autoencoder that measures how well it can rebuild the clip. On a combined dataset of real ship recordings and public background-ocean noise, the paper reports 71.3% accuracy for classifying five classes (background, cargo, passenger ship, tanker, tug) and 91.5% accuracy for novelty detection of a withheld ship type. The authors also report a clear trade-off: classifying the original noisy clips gives 77.1% accuracy, while classifying the denoised clips gives 71.3%, so their case for the pipeline rests on the added novel-event detection, not on a classification record.

What carries the argument

The load-bearing mechanism is the novelty detector: a feedforward autoencoder trained only on background-noise spectrograms, whose reconstruction error becomes an anomaly score, with a cutoff fixed at the 70th percentile of reconstruction errors computed from background-only training samples. Test inputs above that cutoff are flagged as novel. Around this detector sits a U-Net denoiser (the ORCA-CLEAN method) that works on spectrograms, and an ImageNet-pretrained ResNet18 fine-tuned for the five-class ship and background task. The whole chain is evaluated under a strict no-leakage protocol: two-second clips from the same original recording never appear in both training and test sets.

What would settle it

Sweep the detection threshold across percentiles and plot a full ROC curve on held-out background and tug samples, then re-run the fixed 70th-percentile threshold on background recordings from different seasons, depths, or sensor gains and count false positives. If accuracy drops sharply away from the chosen percentile or in new background conditions, the reported generalization is an artifact of calibration rather than a property of the learned representation.

Watch

Extended reading notes

Core claim

The discovery AquaSignal tries to establish is that segmenting, denoising, classifying, and novelty-detecting underwater sound can be chained into one pipeline that works on real, noisy recordings, and that the autoencoder can recognize a withheld ship type (tug) as novel even though that class never appeared during training. On a combined dataset built from the DeepShip benchmark and public background-ocean recordings, the paper reports 71.3% classification accuracy and 91.5% novelty-detection accuracy, with precision, recall, and F1 in the same range. The paper also reports a cost: the classifier reaches 77.1% accuracy on the original noisy clips and 71.3% on denoised clips, so the denoiser removes information the classifier had been using. The authors' case is that the contribution is the combination—the first integrated preprocessing, denoising, classification, and novelty pipeline on this benchmark—rather than a new classification record.

Load-bearing premise

The novelty-detection result assumes that a cutoff fixed at the 70th percentile of reconstruction errors on training background clips will also separate unseen tug sounds from background sounds in the test recordings; if the test background differs from the training background, the 91.5% figure overstates real-world performance.

Editorial extensions

If this is right

  • The 91.5% novelty-detection accuracy implies that a monitoring system can flag a ship type it never trained on by thresholding autoencoder reconstruction error, without any labeled anomalies.
  • The 71.3% classification accuracy on a strict no-leak split implies that realistic evaluations of vessel classifiers are harder than earlier reported numbers suggest, and that accuracy obtained without such splits can be inflated by information leakage.
  • The ablation study implies that under the data-scarce conditions of this benchmark, shallower models (ResNet18 over ResNet34 and DenseNet169, the autoencoder over EfficientNet and a diffusion-based detector) generalize better.
  • The measured drop from 77.1% to 71.3% when classifying denoised audio implies that a denoiser's value must be judged by its effect on downstream classification and detection, not by spectrogram clarity alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the fixed 70th-percentile threshold is an arbitrary calibration point; a threshold chosen by cross-validation, or a full ROC-based decision rule, would probably transfer more reliably to new acoustic environments.
  • Beyond the paper: because real deployment noise changes with season, current, and sensor aging, an online recalibration of the autoencoder threshold would be a natural stress test of the 91.5% figure.
  • Beyond the paper: the denoising cost suggests an adaptive pipeline that classifies both raw and denoised clips and lets the novelty score arbitrate between them could recover some of the lost classification accuracy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces AquaSignal, a four-stage processing pipeline for underwater acoustic signals: preprocessing (2-second, 32 kHz segments from DeepShip and ONC background data), denoising with the ORCA-CLEAN U-Net, classification with a fine-tuned ResNet18, and unsupervised novelty detection with an autoencoder. The reported headline results are 71.3% classification accuracy and 91.5% novelty-detection accuracy on a 25,000-sample curated set with strict train/test separation at the recording level. The paper also claims, in the abstract and introduction, that AquaSignal improves signal clarity and task performance, and it presents ablation tables comparing its classifier and detector against in-house reimplemented baselines.

Significance. An integrated, modular, open-dataset pipeline for denoising, classification, and novelty detection would be useful for maritime monitoring, especially if it can be shown to generalize under a leakage-free partition. The paper also addresses a real gap in the DeepShip literature by considering novelty detection. However, the central performance claims are not supported by the manuscript's own evidence: the denoising stage lowers classification accuracy relative to the same classifier on noisy data, and the novelty-detection metrics are internally inconsistent and below the trivial always-background baseline under the stated setup. As a result, the contribution, as evaluated, does not yet establish the claimed benefit.

major comments (4)
  1. [Section 8, Discussion and Concluding Remarks] The abstract and introduction claim that AquaSignal 'improves signal clarity and task performance,' but Section 8 states that 'when trained on the original noisy samples, the classifier returned an accuracy of 77.1%, whereas when trained on the denoised data, the accuracy drops to 71.3%.' Since the pipeline described in Section 5 feeds denoised spectrograms into the ResNet18 classifier, the end-to-end classification accuracy is 71.3%, six points below the same classifier without denoising. This is an internal contradiction between the stated contribution and the reported evidence. The claim should be revised to state that denoising improves visual clarity but degrades classification accuracy in this setting.
  2. [Section 7, Table 2 and Section 5, Detection] The novelty-detection results are not credible as reported. With 1% tug contamination in the background test set, an always-background classifier achieves 99% accuracy, so the reported 91.5% accuracy is below a trivial baseline. Moreover, the reported precision (87.8%) and recall (81.3%) are mutually inconsistent with 91.5% accuracy under standard definitions: with 1% positives, precision 87.8% and recall 81.3% imply an accuracy of roughly 99.7%. The authors should report the confusion matrix, the true positive/negative counts, and a majority-class baseline.
  3. [Section 5, Detection] The reconstruction-error threshold is set at the 70th percentile of background-only training errors. If the test background distribution resembles the training background distribution, this threshold by construction classifies about 30% of test background samples as novel. With only 1% contamination, the maximum achievable accuracy under that assumption is approximately 70%, not 91.5%. The described methodology is therefore incompatible with the reported accuracy unless the threshold definition is misstated, the test background distribution is substantially different, or the threshold was re-calibrated on test data. This point is load-bearing for the novelty-detection claim and must be clarified and validated on held-out data.
  4. [Section 7, Tables 1 and 2] The ablation comparisons in Tables 1 and 2 are all against models reimplemented in-house under the same pipeline. No error bars, confidence intervals, repeated runs, or tests of statistical significance are provided, and no external published results are used as anchors. The conclusion that AquaSignal 'outperformed the state-of-the-art methods when trained using the same policy' is therefore not established. Reporting variance or a small number of seeds, and comparing against published results under a comparable leakage-free partition, would be necessary to support the comparison.
minor comments (5)
  1. [General] There are several typographical errors that should be corrected, including 'achitecture' in the opening of Section 8, 'datset' in the Introduction, and 'DesNet169' in Section 8.
  2. [References] References [9] and [10] are the same paper (He et al., deep residual learning) and should be merged or renumbered; similarly, [10] and [31] in the original numbering are listed separately but refer to the same work.
  3. [Section 7] The sentence 'The results presented in the 3 section cannot be directly compared with those reported in the 7 section' should be rewritten to refer to Sections 3 and 7 by name rather than by bare numerals.
  4. [Section 5] In the Detection paragraph, 'Following an extensive hyperparameter optimisation and procedure' is ungrammatical and should be rephrased.
  5. [Section 4] The ResNet18 description calls it a '72-layer architecture with 18 deep layers,' which is confusing without a more precise architectural specification; consider stating the standard ResNet18 configuration explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports empirical measurements of a pipeline assembled from external, standard components; no load-bearing step reduces to its own inputs.

full rationale

AquaSignal is an empirical systems paper, not a derivation, so most circularity patterns do not apply. The novelty-detection threshold is described as "a reconstruction error threshold was defined using the 70th percentile of reconstruction error values computed exclusively from background samples," which is standard anomaly-detection calibration on training background data; the reported 91% detection accuracy is then measured on held-out test samples, not forced by the calibration step, so this is not a fitted input disguised as a prediction. The denoising module uses ORCA-CLEAN from Bergler et al. [1], the classifier uses standard ImageNet-pretrained ResNet18 [7], and the autoencoder is a standard feedforward architecture; all are external or conventional components. The ablation tables compare against in-house reimplementations of ResNet34, DenseNet169, EfficientNet, and DDAD under identical conditions; this is a controlled comparison, not circular, though it should not be read as external benchmarking. There are no author self-citations and no imported uniqueness theorem. The Discussion explicitly flags that "when trained on the original noisy samples, the classifier returned an accuracy of 77.1%, whereas when trained on the denoised data, the accuracy drops to 71.3%," which undermines the abstract's claim that AquaSignal improves task performance; that is an internal correctness/consistency problem, not circularity, because the claim is empirically contradicted rather than assumed into existence. Similarly, the reported 91.5% detection accuracy is numerically inconsistent with the accompanying precision (87.8%) and recall (81.3%) under standard definitions for a 1% contamination test set, but that is a reporting or evaluation inconsistency, not circular reasoning. Overall, the paper's central empirical claims are self-contained measurements of standard components, so the circularity score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claims rest on dataset purity, transferability of pretrained models, and the validity of reconstruction error as a novelty score; none of these is independently verified in the paper.

free parameters (5)
  • Reconstruction error threshold = 70th percentile of background training errors
    Chosen on background training samples, not on an independent validation set; results depend on this choice.
  • Samples per class = 5,000 per class (25,000 total)
    Subsampling choice with no justification; affects class balance and evaluation.
  • Segment length and sample rate = 2 seconds at 32 kHz
    Design choice that fixes the input representation; no ablation provided.
  • Train/test split = 90:10 random split
    Split is random without a stated seed, so exact reproduction is impossible.
  • Classifier and autoencoder hyperparameters = Learning rate 0.001, 50 epochs; denoiser 100 epochs
    Reported as tuned but batch size and latent dimension are unspecified.
assumptions (4)
  • domain assumption ImageNet-pretrained ResNet18 features transfer to spectrogram-based underwater acoustic classification
    Fine-tuning ImageNet weights on spectrogram arrays is assumed beneficial without a comparison to training from scratch (Section 5, Classification).
  • domain assumption ONC background recordings contain no vessel signals and represent a clean 'background' class
    The paper adds ONC audio as background without verifying that it is free of ships or other signals (Section 5, Preprocessing).
  • domain assumption ORCA-CLEAN denoising, designed for killer whale calls, generalizes to ship acoustics without clean ground truth
    The model is fine-tuned on ship data, but no objective denoising metric is given (Section 5, Denoising).
  • domain assumption Autoencoder reconstruction error is a valid novelty score for separating unseen vessel types from background
    No evidence shows reconstruction error separates the withheld tug class from background; the reported accuracy is below the majority-class baseline (Section 7).

how reviews work

0 comments
Cite this review

Pith. "Pith review of AquaSignal: An Integrated Framework for Robust Underwater Acoustic Analysis." pith.science (2026). https://pith.science/paper/GGPWSSNT

@misc{pith2026250514285,
  author       = {Pith},
  title        = {Pith review of: AquaSignal: An Integrated Framework for Robust Underwater Acoustic Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GGPWSSNT}},
  note         = {Machine review of arXiv:2505.14285}
}
read the original abstract

This paper presents AquaSignal, a modular and scalable pipeline for preprocessing, denoising, classification, and novelty detection of underwater acoustic signals. Designed to operate effectively in noisy and dynamic marine environments, AquaSignal integrates state-of-the-art deep learning architectures to enhance the reliability and accuracy of acoustic signal analysis. The system is evaluated on a combined dataset from the Deepship and Ocean Networks Canada (ONC) benchmarks, providing a diverse set of real-world underwater scenarios. AquaSignal employs a U-Net architecture for denoising, a ResNet18 convolutional neural network for classifying known acoustic events, and an AutoEncoder-based model for unsupervised detection of novel or anomalous signals. To our knowledge, this is the first comprehensive study to apply and evaluate this combination of techniques on maritime vessel acoustic data. Experimental results show that AquaSignal improves signal clarity and task performance, achieving 71% classification accuracy and 91% accuracy in novelty detection. Despite slightly lower classification performance compared to some state-of-the-art models, differences in data partitioning strategies limit direct comparisons. Overall, AquaSignal demonstrates strong potential for real-time underwater acoustic monitoring in scientific, environmental, and maritime domains.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 29 canonical work pages

  1. [1]

    Bergler et al

    C. Bergler et al. Orca-clean: A deep denoising toolkit for killer whale communication. In INTERSPEECH, pages 1136–1140, 2020

    work page 2020

  2. [2]

    Bueger and T

    C. Bueger and T. Liebetrau. Critical maritime infrastructure protection: What’s the trouble? Marine Policy, 155:105772, 2023. ISSN 0308- 597X

    work page 2023

  3. [3]

    L. C. Domingos et al. A survey of underwater acoustic data classifica- tion methods using deep learning for shoreline surveillance. Sensors, 22(6), 2022

    work page 2022

  4. [4]

    B. G. Ferguson. Defense applications of acoustic signal processing. Acoustics Today, 15(3):10–18, 2019

    work page 2019

  5. [5]

    Gaggero, E

    T. Gaggero, E. Armelloni, A. Codarin, C. Chicco, M. Spoto, C. Fran- zosini, S. Ciriaco, and M. Picciulin. Electric boat underwater radiated noise and its potential impact on species of conservation interest. Ma- rine Pollution Bulletin, 199:115937, 2024

    work page 2024

  6. [6]

    Gao et al

    R. Gao et al. Underwater acoustic signal denoising algorithms: A survey of the state-of-the-art. arXiv e-prints, pages arXiv–2407, 2024

    work page 2024

  7. [7]

    W. Gu, S. Bai, and L. Kong. A review on 2d instance segmentation based on deep neural networks. Image and Vision Computing , 120: 104401, 2022

    work page 2022

  8. [8]

    Gurrola-Ramos, O

    J. Gurrola-Ramos, O. Dalmau, and T. E. Alarcón. A residual dense u- net neural network for image denoising. IEEE Access, 9:31742–31754,

    work page

Show all 34 references
  1. [9]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for im- age recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

    2016

  2. [10]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  3. [11]

    Heesemann et al

    M. Heesemann et al. Ocean networks canada: From geohazards re- search laboratories to smart ocean systems. Oceanography, June 2014

    2014

  4. [12]

    B. M. Howe et al. Observing the oceans acoustically. Frontiers in Marine Science, 6, 2019. ISSN 2296-7745

    2019

  5. [13]

    Huang, Z

    G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), July 2017

    2017

  6. [14]

    Irfan et al

    M. Irfan et al. Deepship: An underwater acoustic benchmark dataset and a separable convolution based autoencoder for classification.Expert Systems with Applications, 183:115270, 2021

    2021

  7. [15]

    Jr et al

    M. Jr et al. Chapter 14 - time-frequency methods in radar, sonar, and acoustics. In B. Boashash, editor, Time-Frequency Signal Analysis and Processing (Second Edition), pages 793–856. Academic Press, Oxford, second edition edition, 2016

    2016

  8. [16]

    Kim et al

    K.-I. Kim et al. A method for underwater acoustic signal classifica- tion using convolutional neural network combined with discrete wavelet transform. International Journal of Wavelets, Multiresolution and Infor- mation Processing, 19(04):2050092, 2021

    2021

  9. [17]

    Lambert, F

    B. Lambert, F. Forbes, A. Tucholka, S. Doyle, H. Dehaene, and M. Do- jat. Trustworthy clinical ai solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis. arXiv preprint arXiv:2210.03736, 2022

    arXiv

  10. [18]

    Li et al

    D. Li et al. Recent advances in acoustic technology for aquaculture: A review. Reviews in Aquaculture, 16(1):357–381, 2024

    2024

  11. [19]

    Mathew, F

    A. Mathew, F. M. Sidhique, P. Jinsha, and S. K. Ambat. Passive under- water target classification using sparse models. In 2023 International Symposium on Ocean Technology (SYMPOL), pages 1–4. IEEE, 2023

    2023

  12. [20]

    Michelucci

    U. Michelucci. An introduction to autoencoders. arXiv preprint arXiv:2201.03898, 2022

    2022 arXiv

  13. [21]

    Misra et al

    S. Misra et al. Multi-channel transfer learning of chest x-ray images for screening of covid-19. Electronics, 9(9):1388, 2020

    2020

  14. [22]

    Mousakhan, T

    A. Mousakhan, T. Brox, and J. Tayyub. Anomaly detection with con- ditioned denoising diffusion models. arXiv preprint arXiv:2305.15956, 2023

    2023 arXiv

  15. [23]

    Müller, J

    N. Müller, J. Reermann, and T. Meisen. Navigating the depths: A comprehensive survey of deep learning for passive underwater acous- tic target recognition. IEEE Access , 12:154092–154118, 2024. doi: 10.1109/ACCESS.2024.3480788

    2024

  16. [24]

    Ren et al

    J. Ren et al. Ualf: A learnable front-end for intelligent underwater acoustic classification system. Ocean Engineering, 264:112394, 2022

    2022

  17. [25]

    T. A. Smith and J. Rigby. Underwater radiated noise from marine ves- sels: A review of noise reduction methods and technology. Ocean En- gineering, 266:112863, 2022

    2022

  18. [26]

    T. A. Smith, A. G. La Rosa, and B. Wood. Underwater radiated noise from small craft in shallow water: Effects of speed and running attitude. Ocean Engineering, 306:118040, 2024

    2024

  19. [27]

    Smookler, B

    M. Smookler, B. Clark, and J. Ostrander. Underwater detection and surveillance technology for commercial port and vessel security. who is going to pay for it? In Proceedings of OCEANS 2005 MTS/IEEE, pages 935–940 V ol. 1, 2005. doi: 10.1109/OCEANS.2005.1639875

    2005

  20. [28]

    Tan and Q

    M. Tan and Q. Le. EfficientNet: Rethinking model scaling for convolu- tional neural networks. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learn- ing, volume 97 of Proceedings of Machine Learning Research , pages 61...

    2019

  21. [29]

    Tian et al

    S.-Z. Tian et al. Joint learning model for underwater acoustic target recognition. Knowledge-Based Systems, 260:110119, 2023

    2023

  22. [30]

    Whata, K

    A. Whata, K. Dibeco, K. Madzima, and I. Obagbuwa. Uncertainty quantification in multi-class image classification using chest x-ray im- ages of covid-19 and pneumonia. Frontiers in Artificial Intelligence, 7: 1410841, 2024

    2024

  23. [31]

    Y . Xie, J. Ren, and J. Xu. Adaptive ship-radiated noise recognition with learnable fine-grained wavelet transform.Ocean Engineering, 265: 112626, 2022

    2022

  24. [32]

    Y . Xie, J. Ren, and J. Xu. Unraveling complex data diversity in under- water acoustic target recognition through convolution-based mixture of experts. Expert Systems with Applications, 249, 2024

    2024

  25. [34]

    J. Xu, Y . Xie, and W. Wang. Underwater acoustic target recognition based on smoothness-inducing regularization and spectrogram-based data augmentation. Ocean Engineering, 281:114926, 2023

    2023

  26. [2021]

    doi: 10.1109/ACCESS.2021.3061062

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.