pith. sign in

arxiv: 2509.21087 · v3 · submitted 2025-09-25 · 📡 eess.AS · cs.LG· cs.SD

Are Modern Speech Enhancement Systems Vulnerable to Adversarial Attacks?

Rostislav Makarov , Lea Sch\"onherr , Timo Gerkmann This is my paper

Pith reviewed 2026-05-18 14:31 UTC · model grok-4.3

classification 📡 eess.AS cs.LGcs.SD
keywords adversarial attacksspeech enhancementpredictive modelsdiffusion modelspsychoacoustic maskingsemantic manipulationmodel robustnessaudio security
0
0 comments X

The pith

Adversarial noise can make speech enhancement systems output entirely different semantic meanings.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows that modern machine learning speech enhancement models are open to adversarial attacks because of how expressive they have become. Attackers can add noise that stays hidden under psychoacoustic masking from the original audio, yet the enhanced output ends up conveying a different meaning. This matters for any system that relies on cleaned-up speech, such as hearing aids or voice interfaces, since a successful attack could change what is understood without anyone noticing the change in the input. The authors confirm the effect through experiments on predictive models and observe that diffusion models using stochastic sampling resist the manipulation by their structure.

Core claim

Advanced speech enhancement models can be susceptible to adversarial attacks. Specifically, adversarial noise, carefully crafted and psychoacoustically masked by the original input, can be injected such that the enhanced speech output conveys an entirely different semantic meaning. Contemporary predictive speech enhancement models can indeed be manipulated in this way, while diffusion models with stochastic samplers exhibit inherent robustness to such adversarial attacks by design.

What carries the argument

Psychoacoustically masked adversarial perturbations applied to the input of predictive speech enhancement models to force a change in the semantic content of the output.

If this is right

  • Predictive speech enhancement models can be made to produce outputs whose meaning differs from the input.
  • The vulnerability stems directly from the models' ability to make large expressive changes to the signal.
  • Diffusion models equipped with stochastic samplers resist these semantic-changing attacks without extra defenses.
  • Applications that depend on accurate speech enhancement face a new class of input-based manipulation risks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Voice assistants and live captioning tools may require separate detection steps for these hidden changes.
  • Hybrid enhancement pipelines could combine predictive speed with diffusion robustness to limit exposure.
  • Evaluating attack success rates across different acoustic environments would clarify how often the effect appears outside labs.

Load-bearing premise

The crafted adversarial perturbations remain effective and imperceptible under real-world acoustic conditions and model deployments beyond the controlled experimental setup.

What would settle it

A recording in which the adversarial perturbation is added to real speech under natural room acoustics, the signal is passed through the enhancement model, and the resulting audio is transcribed to check whether its semantic meaning matches the original or the attacked version.

read the original abstract

Machine learning approaches for speech enhancement are becoming increasingly expressive, enabling ever more powerful modifications of input signals. In this paper, we demonstrate that this expressiveness introduces a vulnerability: advanced speech enhancement models can be susceptible to adversarial attacks. Specifically, we show that adversarial noise, carefully crafted and psychoacoustically masked by the original input, can be injected such that the enhanced speech output conveys an entirely different semantic meaning. We experimentally verify that contemporary predictive speech enhancement models can indeed be manipulated in this way. Furthermore, we highlight that diffusion models with stochastic samplers exhibit inherent robustness to such adversarial attacks by design.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper claims that modern speech enhancement systems based on predictive models are vulnerable to adversarial attacks, where carefully crafted adversarial noise that is psychoacoustically masked by the original input can be injected to make the enhanced speech output convey an entirely different semantic meaning. Experiments verify this manipulation on contemporary predictive models, while diffusion models with stochastic samplers are shown to exhibit inherent robustness by design.

Significance. If the empirical results hold under scrutiny, the work is significant for highlighting a security vulnerability in increasingly expressive ML-based speech enhancement systems used in real-world applications such as voice communication and assistive devices. It provides concrete experimental evidence of semantic manipulation in predictive models and identifies diffusion-based approaches as more robust, which could guide future secure designs. The empirical verification on predictive models is a clear strength.

major comments (2)
  1. Abstract: the claim that adversarial perturbations produce an 'entirely different semantic meaning' in the enhanced output is load-bearing for the central thesis, yet the abstract (and apparent methods) provides no details on attack generation procedure, evaluation metrics for semantic change (e.g., ASR-based transcription differences or semantic similarity), or experimental controls, which prevents assessment of whether the reported verification actually supports the stated result.
  2. Experimental section: the verification is confined to controlled conditions on specific predictive models and datasets; without reported tests for generalization under distribution shifts (different room acoustics, unseen enhancement architectures, or real-world noise profiles), the broader claim that 'contemporary predictive speech enhancement models can indeed be manipulated' risks being limited to the exact experimental setup and does not yet establish practical vulnerability.
minor comments (1)
  1. Figure captions and axis labels could be expanded to explicitly state the models, datasets, and attack parameters used, improving reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and detailed comments. We address each major comment below with clarifications from the manuscript and indicate where revisions have been made to improve clarity and address concerns.

read point-by-point responses

  1. Referee: [—] Abstract: the claim that adversarial perturbations produce an 'entirely different semantic meaning' in the enhanced output is load-bearing for the central thesis, yet the abstract (and apparent methods) provides no details on attack generation procedure, evaluation metrics for semantic change (e.g., ASR-based transcription differences or semantic similarity), or experimental controls, which prevents assessment of whether the reported verification actually supports the stated result.

    Authors: The abstract is intentionally concise as a high-level summary. The attack generation procedure, which optimizes perturbations under psychoacoustic masking constraints to induce semantic divergence, is fully detailed in Section 3 of the manuscript. Semantic change is quantified using ASR transcription differences (via models such as Whisper) and semantic similarity metrics based on sentence embeddings, with controls including clean-input baselines and random-noise comparisons reported in Section 4. To address the concern, we have revised the abstract to briefly reference the evaluation metrics used for semantic manipulation. revision: yes

  2. Referee: [—] Experimental section: the verification is confined to controlled conditions on specific predictive models and datasets; without reported tests for generalization under distribution shifts (different room acoustics, unseen enhancement architectures, or real-world noise profiles), the broader claim that 'contemporary predictive speech enhancement models can indeed be manipulated' risks being limited to the exact experimental setup and does not yet establish practical vulnerability.

    Authors: Our experiments verify the vulnerability on multiple contemporary predictive models using standard datasets, which directly supports the claim that such models can be manipulated. We agree that tests under broader distribution shifts would be valuable for assessing real-world practicality; however, the manuscript's focus is on establishing the existence of the attack vector rather than exhaustive generalization. We have expanded the discussion and limitations sections to explicitly address this scope and potential extensions. revision: partial

Circularity Check

0 steps flagged

No circularity: empirical verification of adversarial attacks on speech enhancement

full rationale

The paper's central claim—that carefully crafted adversarial noise can alter semantic meaning in enhanced speech outputs—is supported solely by experimental verification on contemporary predictive models, with an additional observation that diffusion models exhibit inherent robustness by design. No derivations, equations, fitted parameters, or self-citations are presented that reduce any result to its own inputs by construction. The work is self-contained as an empirical demonstration against external benchmarks, with no load-bearing steps that collapse into self-definition, renaming, or imported uniqueness from prior author work.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available; no explicit free parameters, axioms, or invented entities are described in the provided text.

pith-pipeline@v0.9.0 · 5631 in / 991 out tokens · 40160 ms · 2026-05-18T14:31:41.846989+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 22 canonical work pages · 1 internal anchor

  1. [1]

    A well-known example is the addition of adversarial noise to an image of a panda so that a classifier would detect a gibbon instead [1]

    INTRODUCTION Adversarial attacks are a method where adversarial noise, designed to be hardly perceivable by humans, is added to the data such that the output of a deep neural network (DNN) is yielding a result that is controlled by the attacker but unintended by the user. A well-known example is the addition of adversarial noise to an image of a panda so ...

    work page

  2. [2]

    Are Modern Speech Enhancement Systems Vulnerable to Adversarial Attacks?

    BACKGROUND Speech Enhancement (SE) aims to recover a clean speech spectro- gramSfrom a noisy observationY. We work in the complex short- time Fourier transform (STFT) domain and adopt the standard addi- tive model Y=S+N, S, Y, N∈C F×T , whereNdenotes environmental noise. We consider two broad fam- ilies of SE models.Predictiveapproaches directly mapY7→ ˆS...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  3. [3]

    ADVERSARIAL A TTACKS We consider targetedwhite-boxattacks: the attacker knows the en- hancement system and can backpropagate through it down to the input. Given a source mixtureY user =S user +Nand a speech signal of comparable length targeted by the attackerS attacker in the complex STFT domain, the goal is to add a small perturbation δ∈C F×T to the sour...

    work page

  4. [4]

    Model preparation We adopt the SGMSE+pipeline3 as our baseline repository to obtain a strong diffusion-based SE system

    EXPERIMENTAL SETUP 4.1. Model preparation We adopt the SGMSE+pipeline3 as our baseline repository to obtain a strong diffusion-based SE system. We prepared a generative dif- fusion model and two predictive baselines that reuse the same back- bone but differ only in how ˆSis produced (Section 2). We took the NCSN++ U-Net backbone as a baseline architecture...

    work page

  5. [5]

    Below we highlight the main results

    RESULTS AND ANALYSIS Table 1 summarizes targeted attacks on 100 EARS-WHAM-v2 pairs. Below we highlight the main results. Direct Map (predictive).Unconstrained attacks steer almost perfectly (e.g., WER≈0.02, AS-ESTOI≈0.94) while being clearly audible SNR≈-2.9 dB. Introducing constraints exposes a clean two–knob trade-off: (1) Fixed energy (ε=10), sweepλ:wi...

    work page

  6. [6]

    That is, by adding adversarial noise, an attacker can trick a speech enhancement system to output a completely different signal than intended by the user

    CONCLUSIONS In this paper, we show that modern speech enhancement systems exhibit some vulnerability to adversarial attacks. That is, by adding adversarial noise, an attacker can trick a speech enhancement system to output a completely different signal than intended by the user. To that end, we build white-box attacks with psychoacoustic masking and anℓ 2...

    work page

  7. [7]

    ACKNOWLEDGEMENTS Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – 545210893, 498394658. The authors grate- fully acknowledge the scientific support and HPC resources pro- vided by the Erlangen National High Performance Computing Cen- ter (NHR@FAU) of the Friedrich-Alexander-Universit ¨at Erlangen- N¨urnberg (FAU) under the...

    work page

  8. [8]

    Explaining and harnessing adversarial examples,

    Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy, “Explaining and harnessing adversarial examples,” inInt. Conf. on Learning Representations (ICLR), San Diego, CA, USA, 2015

    work page 2015

  9. [9]

    Audio adversarial exam- ples: Targeted attacks on speech-to-text,

    Nicholas Carlini and David Wagner, “Audio adversarial exam- ples: Targeted attacks on speech-to-text,” inIEEE security and privacy workshops (SPW), San Francisco, CA, USA, 2018

    work page 2018

  10. [10]

    Adversarial attacks against automatic speech recognition systems via psychoacoustic hiding,

    Lea Sch ¨onherr, Katharina Kohls, Steffen Zeiler, Thorsten Holz, and Dorothea Kolossa, “Adversarial attacks against automatic speech recognition systems via psychoacoustic hiding,” inNet- work and Distributed System Security Symposium (NDSS), San Diego, CA, USA, 2019

    work page 2019

  11. [11]

    Richard C. Hendriks, Timo Gerkmann, and Jesper Jensen, DFT-Domain Based Single-icrophone Noise Reduction for Speech Enhancement: A Survey of the State-of-the-Art, Num- ber 11 in Synthesis Lectures on Speech and Audio Processing. Morgan & Claypool, Williston, VT, 2013

    work page 2013

  12. [12]

    A regres- sion approach to speech enhancement based on deep neural networks,

    Yong Xu, Jun Du, Li-Rong Dai, and Chin-Hui Lee, “A regres- sion approach to speech enhancement based on deep neural networks,”IEEE/ACM Trans. on Audio, Speech, and Language Proc. (TASLP), vol. 23, no. 1, pp. 7–19, 2015

    work page 2015

  13. [13]

    A fully convolutional neural network for speech enhancement,

    Se Rim Park and Jinwon Lee, “A fully convolutional neural network for speech enhancement,” inISCA Interspeech, Stock- holm, Sweden, 2017

    work page 2017

  14. [14]

    A fully convolutional neural network for com- plex spectrogram processing in speech enhancement,

    Zhiheng Ouyang, Hongjiang Yu, Wei-Ping Zhu, and Benoit Champagne, “A fully convolutional neural network for com- plex spectrogram processing in speech enhancement,” inIEEE Int. Conf. on Acoustics, Speech and Signal Proc. (ICASSP), Brighton, UK, 2019

    work page 2019

  15. [15]

    Phase-aware speech en- hancement with deep complex U-Net,

    Hyeong-Seok Choi, Jang-Hyun Kim, Jaesung Huh, Adrian Kim, Jung-Woo Ha, and Kyogu Lee, “Phase-aware speech en- hancement with deep complex U-Net,” inInt. Conf. on Learn- ing Representations (ICLR), Vancouver, Canada, 2018

    work page 2018

  16. [16]

    Complex ratio masking for monaural speech separation,

    Donald S. Williamson, Yuxuan Wang, and DeLiang Wang, “Complex ratio masking for monaural speech separation,” IEEE/ACM Trans. on Audio, Speech, and Language Proc. (TASLP), vol. 24, no. 3, pp. 483–492, 2016

    work page 2016

  17. [17]

    Score-based generative modeling through stochastic differential equations,

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole, “Score-based generative modeling through stochastic differential equations,” inInt. Conf. on Learning Representations (ICLR), Vienna, Austria, 2021

    work page 2021

  18. [18]

    Speech enhancement and derever- beration with diffusion-based generative models,

    Julius Richter, Simon Welker, Jean-Marie Lemercier, Bunlong Lay, and Timo Gerkmann, “Speech enhancement and derever- beration with diffusion-based generative models,”IEEE/ACM Trans. on Audio, Speech, and Language Proc. (TASLP), vol. 31, pp. 2351–2364, 2023

    work page 2023

  19. [19]

    Towards deep learning models resistant to adversarial attacks,

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu, “Towards deep learning models resistant to adversarial attacks,” inInt. Conf. on Learn- ing Representations (ICLR), Vancouver, Canada, 2018

    work page 2018

  20. [20]

    EARS: An anechoic fullband speech dataset benchmarked for speech enhancement and dereverberation,

    Julius Richter, Yi-Chiao Wu, Steven Krenn, Simon Welker, Bunlong Lay, Shinjii Watanabe, Alexander Richard, and Timo Gerkmann, “EARS: An anechoic fullband speech dataset benchmarked for speech enhancement and dereverberation,” in ISCA Interspeech, Kos Island, Greece, 2024

    work page 2024

  21. [21]

    Distillation and pruning for scalable self-supervised representation-based speech qual- ity assessment,

    Benjamin Stahl and Hannes Gamper, “Distillation and pruning for scalable self-supervised representation-based speech qual- ity assessment,” inIEEE Int. Conf. on Acoustics, Speech and Signal Proc. (ICASSP), Hyderabad, India, 2025

    work page 2025

  22. [22]

    Robust speech recognition via large-scale weak supervision,

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever, “Robust speech recognition via large-scale weak supervision,” inInt. Conf. on Learning Representations (ICLR), Kigali, Rwanda, 2023, PMLR

    work page 2023