REVIEW 4 major objections 4 minor 44 references
Imperio: Robust Over-the-Air Adversarial Examples for Automatic Speech Recognition Systems
T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A crafted audio clip can spoof speech recognition in any room
desk verdict A genuine over-the-air attack on Kaldi with real-room transfer, but the generic-room claim is only thinly supported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is an additional differentiable layer prepended to the recognizer's network that performs convolution with a room impulse response $h$, so gradient descent can update the raw audio directly. Each step samples a fresh RIR from a distribution $H_{\theta}$ over room parameters—cuboid dimensions, reverberation time $T_{60}$, and source/receiver positions—drawn uniformly over chosen ranges, and the attack maximizes the expected probability of the target transcription $\mathbb{E}_{h\sim H_{\theta}}[P(y' \mid x*h)]$. This is the audio analogue of expectation over transformation in the visual domain. The layer is differentiable because $\partial x_h(n)/\partial x(m) = h(n-m)$. Optionally, psychoacoustic hearing thresholds restrict the perturbation to time-frequency regions where humans cannot perceive it.
What would settle it
Play a batch of generic adversarial examples generated with the paper's distribution in a set of real rooms whose dimensions, reverberation times, or geometry fall outside the simulator's ranges—a car interior, a staircase, an outdoor courtyard, a heavily furnished living room—and count how many produce the exact target transcription; if none achieve zero word error rate while examples in the three tested rooms do, the claim of room independence is refuted.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that robustness to unknown rooms can be achieved by maximizing the targeted recognition probability in expectation over room impulse responses (RIRs) sampled from a simulator, not over the real room. The audio signal is perturbed so that, after convolution with virtually any plausible RIR, the recognizer outputs the attacker's target transcription. In the authors' experiments with Kaldi, generic adversarial examples computed this way achieved zero word error rate in real playback tests across several rooms and distances, including setups with no direct line-of-sight between loudspeaker and microphone, and they outperformed examples tailored to a specific room. This is presented as the first over-the-air attack that is both targeted and independent of the environment.
Load-bearing premise
The room-independence claim rests on the assumption that room impulse responses sampled from the simulator—uniform draws over cuboid room sizes, reverberation times, and source/receiver positions—are representative enough of real rooms that optimizing against them transfers to physical playback.
Editorial extensions
If this is right
- An attacker can create a targeted adversarial audio file offline from the ASR system's parameters alone, play it in an arbitrary room, and have it recognized as the chosen sentence without measuring that room.
- One generic adversarial example covers a range of rooms and recording conditions, so the same file can be replayed in different locations; no physical access or special hardware is needed at the attack site.
- The attack remains possible when the loudspeaker and microphone have no direct line of sight, meaning the command can be hidden behind furniture or outside a door.
- Using psychoacoustic hearing thresholds, the added noise can be shifted below human audibility, although this reduces the success rate.
- Because the optimization only needs backpropagation, the authors argue the approach should transfer to end-to-end speech recognition systems as well.
Reading between the lines
- The reported success rates (roughly 0 to 5 of 20 examples per condition) suggest that in practice an attacker would need to generate many candidate examples and keep only those that pass a simulated over-the-air check; the paper's threat model already allows this batch approach.
- The comparison showing generic examples outperforming room-adapted ones hints that optimizing over a broad RIR distribution acts as a regularizer that prevents overfitting to a single measured impulse response; this could be tested directly across a larger set of real rooms.
- The paper only validates on three ordinary indoor rooms, so the boundary of the any-room claim remains untested; rooms far outside the simulator's parameter ranges—cars, corridors, outdoors—could break the transfer.
- A natural extension is to map how success rate and noise level vary with the acoustic distance between the original utterance and the target transcription; long or phonetically dissimilar targets likely need more perturbation, which would change the real-world threat assessment.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents Imperio, a white-box targeted attack against the Kaldi hybrid DNN-HMM speech recognizer. The attack optimizes the audio signal under an expectation over room impulse responses (RIRs), Eq. (6), implemented by inserting a differentiable convolution layer before the recognizer's feature extraction. RIRs are sampled from a uniform distribution over cuboid room parameters, reverberation time, and source/receiver positions (Section 3.2). The authors also implement a psychoacoustic variant that constrains perturbations to time-frequency regions below hearing thresholds. Evaluation consists of playing optimized examples through a loudspeaker in a laboratory and in three additional rooms, with and without direct line-of-sight, at varying distances and reverberation times. The central claim is that this is the first algorithm producing generic adversarial examples that remain robust over the air without any knowledge of the target room.
Significance. If the room-independence claim were fully established, the paper would be a meaningful advance: it would remove the need for room-specific measurement in over-the-air audio attacks and would extend robustness to a hybrid ASR architecture rather than only end-to-end systems. The paper has real strengths: the RIR-convolution layer in Eqs. (7)-(8) is a clean and differentiable formulation of the physical channel; the evaluation genuinely includes over-the-air playback in held-out rooms; and the no-line-of-sight experiments test a practically important attack condition. These strengths make the work a useful existence proof that some simulated-RIR-hardened examples transfer to real rooms. However, the headline claims about "arbitrary room setups," "any target transcription," and "inconspicuous" examples outrun the reported evidence. The success counts are small, the generic-versus-adapted comparison in Table 5 is undermined by a distribution mismatch, and the evaluation does not vary the target transcription. The paper is therefore a promising demonstration whose scope needs to be narrowed or whose evidence needs to be substantially expanded.
major comments (4)
- [Section 4.3.5, Table 5] The comparison between the generic attack and the "adapted" attack is invalid because the adapted RIR distribution H_theta_adp does not actually contain the real laboratory room. Section 4.3 states that the lab has dimensions b = [8 m, 7 m, 2.8 m], while Table 1 gives H_theta_adp a height range of 3.0 m to 5.0 m. The optimization in Eq. (6) samples RIRs from H_theta_adp, so the adapted attack is optimized over rooms whose ceiling height excludes the test room. For the same reason, H_theta_adp does not cover the T60 = 0.65 s condition used elsewhere in Table 2. Consequently, the conclusion in Section 4.3.5 that "the generic version clearly outperforms the adapted versions" is not supported; the observed difference may simply reflect a misspecified adapted distribution. Please rerun the adapted condition with H_theta_adp ranges that include the measured room geometry and reverberation times, or remove the comparison.
- [Section 4.3.6, Table 6] The central room-transfer claim is supported by very low per-example success rates: 2/20, 1/20, and 1/20 with line-of-sight, and 0/20, 0/20, and 1/20 without line-of-sight across the three test rooms. Given the threat model in Section 3.1, which explicitly allows the attacker to generate many candidates and select robust ones, the existence of a few 0% WER examples is meaningful. However, the paper's stronger claim that a generic example "covers a wide range of rooms" is not established by these numbers. Please report batch-level statistics, such as the distribution of the number of generated examples needed to obtain one transferable example per room, or per-example success probabilities with confidence intervals. Without such a statistic, the results are consistent with lucky selection rather than with coverage of the real-room manifold by H_theta_gen.
- [Section 4.3, metrics paragraph] The manuscript says that adversarial examples that clipped too much were removed from the evaluation, but it never reports how many examples were removed. Every table reports a denominator of 20, yet the effective sample is 20 minus the number of clipped examples. If clipping is frequent or concentrated in certain conditions, the reported success counts overstate the attack's reliability. Please report the number of removed examples per cell in Tables 2-6, and state whether the clipping criterion was applied before or after playback.
- [Abstract, Section 3, and Section 4.3.4] The abstract and conclusion claim that the attack can create "inconspicuous adversarial examples for any target transcription," but the experiments appear to use a single target phrase (visible in Figures 1 and 2) and do not vary the target transcription. No human-listening study is reported; the only evidence regarding inconspicuousness is SNRseg and the use of hearing thresholds. Please either add a modest ablation varying target transcriptions and a perceptual evaluation, or revise the claims to state explicitly that only a single target phrase was tested and that inconspicuousness is supported only by objective psychoacoustic metrics.
minor comments (4)
- [Throughout] There are several typographical errors in names and numbers: "Szuley and Kolter" should be "Szurley and Kolter," "Mittael" should be "Mittal," and the entry "8,7 %" in Table 6 should be "8.7 %."
- [Algorithm 1] The role of h_test and DNN0 in Algorithm 1 is confusing. The text says h_test can be drawn once from H_theta, but it is not clear whether h_test is used only for early stopping or also for the final reported WER. Please clarify that h_test is a single simulated RIR used only for verification during optimization and that final evaluation is performed on recordings of physical playback.
- [Section 4.3.2, Table 3] Table 3 reports hearing-threshold results only for T60 = 0.42 s, while the text says the same experiments as Table 2 were conducted with hearing thresholds. If the other reverberation times were tested, they should be reported; if not, the text should say explicitly that only the shortest reverberation time was used.
- [Reproducibility] The paper mentions a demonstration website with audio examples but does not release code or the RIR sampling implementation. Given the small number of reported successes and the sensitivity of the results to the exact optimization setup, releasing code would materially strengthen reproducibility.
Circularity Check
No significant circularity: the over-the-air transfer claim is verified against held-out real-room recordings, and the optimization objective is not recycled as the outcome metric.
full rationale
The paper's central claim, that generic adversarial examples computed from simulated room impulse responses transfer to real rooms, is empirically evaluated against held-out physical recordings (Tables 2, 3, and 6). Equation (6) is an optimization objective over a distribution of simulated RIRs, and the final success metric is the WER of audio recorded in real rooms, which is external to that optimization. The uniform parameter ranges in Table 1 are chosen a priori to represent generic rooms; they are not fitted to the test rooms, so no fitted input is being renamed as a prediction. The only self-citations, primarily to Schönherr et al. [29] for the direct-feed attack and psychoacoustic hearing thresholds, supply algorithmic building blocks rather than the load-bearing room-independence claim. No uniqueness theorem or ansatz is imported from the authors' prior work: the RIR image-method simulator [2] and the expectation-over-transformation framework [4] are external, and the paper does not claim to derive the transfer result from them by definition. The limited number of real test rooms and low per-example success rates are a legitimate concern about the strength and coverage of the empirical claim, but they are not a form of circularity: the outcome is measured, not assumed.
Assumptions & free parameters
free parameters (5)
- Generic RIR sampling distribution H_theta_gen =
bx, by, bz: 2-15 m, 2-15 m, 2-5 m; T60: 0-1 s
- Adapted RIR distribution H_theta_adp =
bx: 6-10 m, by: 5-9 m, bz: 3-5 m; T60: 0.2-0.6 s
- RIR length M =
512, 1024, or 8192 taps
- Maximum iterations G and RIR resampling interval Q =
G=2000, Q=10
- Learning rate alpha =
not reported
assumptions (5)
- domain assumption Over-the-air transmission is modeled as convolution with a linear time-invariant room impulse response (Eq. 4).
- domain assumption Image-method simulated RIRs with uniform parameter ranges are representative of real rooms.
- domain assumption Kaldi's feature extraction and acoustic DNN are differentiable and available for white-box backpropagation.
- domain assumption Psychoacoustic hearing thresholds (Zwicker and Fastl) predict human imperceptibility of the added perturbation.
- standard math Discrete convolution, the chain rule, and backpropagation are valid for the augmented network.
Cite this review
Pith. "Pith review of Imperio: Robust Over-the-Air Adversarial Examples for Automatic Speech Recognition Systems." pith.science (2026). https://pith.science/paper/IWIB5LGY
@misc{pith2026190801551,
author = {Pith},
title = {Pith review of: Imperio: Robust Over-the-Air Adversarial Examples for Automatic Speech Recognition Systems},
year = {2026},
howpublished = {\url{https://pith.science/paper/IWIB5LGY}},
note = {Machine review of arXiv:1908.01551}
}
read the original abstract
Automatic speech recognition (ASR) systems can be fooled via targeted adversarial examples, which induce the ASR to produce arbitrary transcriptions in response to altered audio signals. However, state-of-the-art adversarial examples typically have to be fed into the ASR system directly, and are not successful when played in a room. The few published over-the-air adversarial examples fall into one of three categories: they are either handcrafted examples, they are so conspicuous that human listeners can easily recognize the target transcription once they are alerted to its content, or they require precise information about the room where the attack takes place, and are hence not transferable to other rooms. In this paper, we demonstrate the first algorithm that produces generic adversarial examples, which remain robust in an over-the-air attack that is not adapted to the specific environment. Hence, no prior knowledge of the room characteristics is required. Instead, we use room impulse responses (RIRs) to compute robust adversarial examples for arbitrary room characteristics and employ the ASR system Kaldi to demonstrate the attack. Further, our algorithm can utilize psychoacoustic methods to hide changes of the original audio signal below the human thresholds of hearing. In practical experiments, we show that the adversarial examples work for varying room setups, and that no direct line-of-sight between speaker and microphone is necessary. As a result, an attacker can create inconspicuous adversarial examples for any target transcription and apply these to arbitrary room setups without any prior knowledge.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Hadi Abdullah, Washington Garcia, Christian Peeters, Patrick Traynor, Kevin R. B. Butler, and Joseph Wilson. 2019. Practical Hidden Voice Attacks against Speech and Speaker Recognition Systems. In Network and Distributed System Security Symposium (NDSS)
work page 2019
-
[2]
Jont B. Allen and David A. Berkley. 1979. Image method for efficiently simulating small-room acoustics. The Journal of the Acoustical Society of America 65, 4 (1979), 943–950
work page 1979
-
[3]
Moustafa Alzantot, Bharathan Balaji, and Mani Srivastava. 2018. Did you hear that? Adversarial examples against automatic speech recognition. arXiv preprint arXiv:1801.00554 (2018)
arXiv 2018
-
[4]
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. 2017. Synthe- sizing Robust Adversarial Examples. CoRR abs/1707.07397 (July 2017), 1–18
arXiv 2017
-
[5]
Mitali Bafna, Jack Murtagh, and Nikhil Vyas. 2018. Thwarting Adversarial Exam- ples: An𝐿1-Robust Sparse Fourier Transform. In Advances in Neural Information Processing Systems 31. 10075–10085
work page 2018
-
[6]
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, and Aleksander Madry. 2019. On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705 (2019)
arXiv 2019
-
[7]
Nicholas Carlini, Pratyush Mishra, Tavish Vaidya, Yuankai Zhang, Micah Sherr, Clay Shields, David A. Wagner, and Wenchao Zhou. 2016. Hidden Voice Com- mands. In USENIX Security Symposium. USENIX, 513–530
work page 2016
-
[8]
Nicholas Carlini and David Wagner. 2017. Towards Evaluating the Robustness of Neural Networks. In Symposium on Security and Privacy . IEEE, 39–57
work page 2017
Show all 44 references
-
[9]
Nicholas Carlini and David Wagner. 2018. Audio adversarial examples: Targeted attacks on speech-to-text. (2018), 1–7
2018
-
[10]
Tao Chen, Longfei Shangguan, Zhenjiang Li, and Kyle Jamieson. 2020. Meta- morph: Injecting Inaudible Commands into Over-the-air Voice Controlled Sys- tems. (2020)
2020
-
[11]
Yuxuan Chen, Xuejing Yuan, Jiangshan Zhang, Yue Zhao, Shengzhi Zhang, Kai Chen, and XiaoFeng Wang. 2020. Devil’s Whisper: A General Approach for Physical Adversarial Attacks against Commercial Black-box Speech Recognition Devices. In USENIX Security Symposium. USENIX
2020
-
[12]
Moustapha Cisse, Yossi Adi, Natalia Neverova, and Joseph Keshet. 2017. Houdini: Fooling Deep Structured Prediction Models. CoRR abs/1707.05373 (July 2017), 1–12
2017 arXiv
-
[13]
Sina Däubener, Lea Schönherr, Asja Fischer, and Dorothea Kolossa. 2020. Detect- ing Adversarial Examples for Speech Recognition via Uncertainty Quantification. arXiv preprint arXiv:2005.14611 (2020)
2020 arXiv
-
[14]
Ivan Evtimov, Kevin Eykholt, Earlence Fernandes, Tadayoshi Kohno, Bo Li, Atul Prakash, Amir Rahmati, and Dawn Song. 2017. Robust Physical-World Attacks on Machine Learning Models. CoRR abs/1707.08945 (July 2017), 1–11
2017 arXiv
-
[15]
Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning. 1050–1059
2016
-
[16]
Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, et al
-
[17]
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. 2018. Black-box Adversarial Attacks with Limited Queries and Information. CoRR abs/1804.08598 (April 2018), 1–10
2018 arXiv
-
[18]
ISO. 1993. Information Technology – Coding of Moving Pictures and Associated Audio for Digital Storage Media at Up to 1.5 Mbits/s – Part3: Audio . ISO 11172-3. International Organization for Standardization
1993
-
[19]
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. InAdvances in Neural Information Processing Systems . 6402–6413
2017
-
[20]
Juncheng Li, Shuhui Qu, Xinjian Li, Joseph Szurley, J Zico Kolter, and Florian Metze. 2019. Adversarial Music: Real World Audio Adversary Against Wake- word Detection System. In Advances in Neural Information Processing Systems (NeurIPS). 11908–11918
2019
-
[21]
Christos Louizos and Max Welling. 2016. Structured and efficient variational deep learning with matrix gaussian posteriors. In International Conference on Machine Learning. 1708–1716
2016
-
[22]
Christoph Lüscher, Eugen Beck, Kazuki Irie, Markus Kitza, Wilfried Michel, Albert Zeyer, Ralf Schlüter, and Hermann Ney. 2019. RWTH ASR systems for LibriSpeech: Hybrid vs Attention. Proceedings of Interspeech (2019), 231–235
2019
-
[23]
Gonzalo Navarro. 2001. A Guided Tour to Approximate String Matching.Comput. Surveys 33, 1 (March 2001), 31–88
2001
-
[24]
Berkay Celik, and Ananthram Swami
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. 2017. Practical Black-Box Attacks Against Machine Learning. In Asia Conference on Computer and Communications Security (ASIA CCS). ACM, 506–519
2017
-
[25]
McDaniel, and Ian J
Nicolas Papernot, Patrick D. McDaniel, and Ian J. Goodfellow. 2016. Trans- ferability in Machine Learning: From Phenomena to Black-Box Attacks using Adversarial Samples. CoRR abs/1605.07277 (May 2016), 1–13
2016 arXiv
-
[26]
Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, Jan Silovsky, Georg Stemmer, and Karel Vesely. 2011. The Kaldi Speech Recogni- tion Toolkit. In Workshop on Automatic Speech R...
2011
-
[27]
Yao Qin, Nicholas Carlini, Ian Goodfellow, Garrison Cottrell, and Colin Raffel
-
[28]
Nirupam Roy, Haitham Hassanieh, and Romit Roy Choudhury. 2017. BackDoor: Making Microphones Hear Inaudible Sounds. In Conference on Mobile Systems, Applications, and Services. ACM, 2–14
2017
-
[29]
Lea Schönherr, Katharina Kohls, Steffen Zeiler, Thorsten Holz, and Dorothea Kolossa. 2019. Adversarial Attacks Against Automatic Speech Recognition Sys- tems via Psychoacoustic Hiding. In Network and Distributed System Security Symposium (NDSS)
2019
-
[30]
Senthil Mani Shreya Khare, Rahul Aralikatte. 2019. Adversarial Black-Box Attacks on Automatic Speech Recognition Systems using Multi-Objective Evolutionary Optimization. Proceedings of Interspeech (2019)
2019
-
[31]
Liwei Song and Prateek Mittal. 2017. Inaudible Voice Commands. CoRR abs/1708.07238 (Aug. 2017), 1–3
2017 arXiv
-
[32]
Joseph Szurley and J Zico Kolter. 2019. Perceptual Based Adversarial Audio Attacks. arXiv preprint arXiv:1906.06355 (2019)
2019 arXiv
-
[33]
Rohan Taori, Amog Kamsetty, Brenton Chu, and Nikita Vemuri. 2018. Targeted adversarial examples for black box audio systems.arXiv preprint arXiv:1805.07820 (2018)
2018 arXiv
-
[34]
Reiter, and Thomas Ristenpart
Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart
-
[35]
Stephen Voran and Connie Sholl. 1995. Perception-based Objective Estimators of Speech. In IEEE Workshop on Speech Coding for Telecommunications. IEEE, 13–14
1995
-
[36]
Binghui Wang and Neil Zhenqiang Gong. 2018. Stealing Hyperparameters in Machine Learning. In Symposium on Security and Privacy . IEEE
2018
-
[37]
Hiromu Yakura and Jun Sakuma. 2019. Robust audio adversarial example for a physical attack. arXiv preprint arXiv:1810.11793 (2019)
2019 arXiv
-
[38]
Wonho Yang. 1999. Enhanced Modified Bark Spectral Distortion (EMBSD): an Objective Speech Quality Measrure Based on Audible Distortion and Cognition Model. Ph.D. Dissertation. Temple University Graduate Board. ACSAC 2020, December 7–11, 2020, Austin, USA Lea Schönherr, Thorste...
1999
-
[39]
Xuejing Yuan, Yuxuan Chen, Yue Zhao, Yunhui Long, Xiaokang Liu, Kai Chen, Shengzhi Zhang, Heqing Huang, Xiaofeng Wang, and Carl A. Gunter. 2018. Com- manderSong: A Systematic Approach for Practical Adversarial Voice Recognition. arXiv preprint arXiv:1801.08535 (2018)
2018 arXiv
-
[40]
Guoming Zhang, Chen Yan, Xiaoyu Ji, Tianchen Zhang, Taimin Zhang, and Wenyuan Xu. 2017. DolphinAttack: Inaudible Voice Commands. In Conference on Computer and Communications Security (CCS) . ACM, 103–117
2017
-
[41]
Eberhard Zwicker and Hugo Fastl. 2007. Psychoacoustics: Facts and Models (third ed.). Springer. Imperio: Robust Over-the-Air Adversarial Examples for Automatic Speech Recognition Systems ACSAC 2020, December 7–11, 2020, Austin, USA A ROOM LAYOUT PLANS Table 7: Microphone and S...
2007
-
[2014]
arXiv preprint arXiv:1412.5567 (2014)
Deep speech: Scaling up end-to-end speech recognition. arXiv preprint arXiv:1412.5567 (2014)
2014 arXiv
-
[2016]
In USENIX Security Symposium
Stealing Machine Learning Models via Prediction APIs. In USENIX Security Symposium. USENIX, 601–618
-
[2019]
In arXiv preprint arXiv:1903.10346
Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition. In arXiv preprint arXiv:1903.10346
1903 arXiv
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.