Pith. sign in

REVIEW 5 major objections 6 minor 16 references

SyntheticPop: Attacking Speaker Verification Systems With Synthetic VoicePops

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 90 Hz sine wave added to 20% of spoofed training audio collapses VoicePop accuracy to 14% and lets over 95% of fake audio pass.

desk verdict A 90 Hz tone can break a reimplementation of VoicePop, but the headline numbers are inflated by class imbalance and post-hoc threshold selection—worth a major revision, not a desk reject. read the letter →

arxiv 2502.09553 v1 pith:BXJJN5VP submitted 2025-02-13 cs.CR cs.LG

Eshaq Jamdar , Amith Kamath Belman This is my paper
classification cs.CRcs.LG
keywords voiceauthenticationspeakerverificationdatapoisoningSyntheticPoppop-noisedetectionGFCCfeaturesSVMclassificationASVSpoof2019
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that VoicePop, a voice-authentication defense that checks the unique 'pop' bursts in how people pronounce certain phonemes, can be defeated by a training-time attacker who knows that feature. The attack, called SyntheticPop, superimposes a 90 Hz sine wave over one fifth of the spoofed audio files in a poisoned training set. In the authors' replica of the VA+VoicePop system, this drops accuracy to 14.01%, compared with 37.21% under a same-rate label-flipping attack, and makes the model classify 95.88% of fake samples as real while still accepting all genuine samples. If that transfers to a real deployment, the pop-noise liveness cue is attacker-controllable, and voice-authentication defenses built on it need to be tested against feature-space poisoning, not only replay and label-corruption attacks.

What carries the argument

The load-bearing object is the synthetic pop signal, a continuous 90 Hz sine wave with amplitude 0.5 added to the whole audio clip. It is calibrated to VoicePop's operating range: pop noises are energy bursts at or below roughly 100 Hz, located by a short-time Fourier transform, then characterized by Gammatone Frequency Cepstral Coefficient features plus two energy-transition deltas ($\delta_1$, the magnitude change from pop to background, and $\delta_2$, the rate of change over the signal). By filling fake training files with this uniform low-frequency pattern, the attacker inserts a feature that the pop-noise stage will extract as if it were a genuine liveness cue, and the SVM absorbs that association during training. The paper uses a same-rate label-flipping attack as the control to show the effect is not just any 20% training corruption but specifically the pop-shaped perturbation.

What would settle it

Run the same 20% SyntheticPop poisoning against the original VoicePop system with its own enrollment data and pop-localization code: if accuracy does not fall to about 14% or if fewer than 95% of poisoned fakes are accepted as real, the claimed success is an artifact of the replica. A second check is to strip the pop-detection and feature-extraction stage from the poisoned classifier and see whether the accuracy loss disappears, which would confirm the effect runs through the pop feature.

Watch

Extended reading notes

Core claim

The paper's central claim is that VA+VoicePop's pop-noise liveness detection is vulnerable to a data-poisoning attack that imitates its own target feature. Under a white-box assumption, the attack adds a 90 Hz, amplitude-0.5 sine wave, $P(t)=0.5\sin(2\pi \cdot 90 \cdot t)$, across the entire duration of randomly selected spoofed training clips (Algorithm 2). Because VoicePop detects pops below about 100 Hz and then classifies acoustic features with an SVM, the poisoned clips carry a clean, uniform pop-like signature; the SVM learns to associate that signature with the 'real' class. At evaluation, the model labels 95.88% of fake samples as real while keeping its true-negative rate at 100%, and overall accuracy falls from the 37.52% even-training baseline to 14.01%. The paper contrasts this with label flipping at the same 20% poison rate, which changes accuracy by only about 0.3 percentage points, and reads the gap as evidence that the vulnerability is specific to VoicePop's pop feature rather than a general property of poisoned SVMs.

Load-bearing premise

The self-built replica of VA+VoicePop, assembled from a different starter codebase and the ASVSpoof 2019 dataset because the original code and data were not published, is faithful enough that the attack's measured 14% accuracy and 95.88% fake-acceptance rate would hold on the real system.

Editorial extensions

If this is right

  • A poisoning attacker who can control 20% of a training set can make VA+VoicePop accept over 95% of fake voice samples while still accepting all genuine users, so the defense no longer separates real from spoofed audio.
  • Because the poison is a generic low-frequency tone, the attack targets the pop-noise feature itself, so defenses that rely on pop-burst shape, amplitude, or timing must treat that feature as attacker-controllable.
  • The paper's full-training result of 69.29% accuracy with 31.31% of fakes classified as real indicates the replay-focused VoicePop design already struggles with logical and deepfake attacks before any poisoning is applied.
  • A 20% label-flipping attack changes accuracy by only about 0.3 percentage points relative to the even-training baseline, so label flipping alone is a weak proxy for evaluating this defense's resilience.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same 90 Hz tone would likely transfer to the original VoicePop system if its pop-localization thresholds are similar, but that is not tested here; applying SyntheticPop to the original enrollment data would settle the transfer.
  • Editorial inference: a defense that rejects whole-file constant tones, for example by requiring pops to coincide with phoneme boundaries or by checking spectral flatness in the low-frequency band, would plausibly resist this specific poison and could be tested with the same pipeline.
  • Editorial inference: because the paper poisons the entire file rather than only phoneme-aligned locations, a targeted variant that injects the sine wave only at pop positions might achieve a similar effect with a smaller poison budget, or might be needed to evade a detector that learns to ignore constant background tones.
  • Editorial inference: the exact percentages (14%, 95.88%) are tied to the replica built from ASVSpoof 2019 and a separate starter codebase; the transferable claim is the mechanism that a pop-shaped poison can redirect the SVM, not the precise numbers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes SyntheticPop, a data-poisoning attack that adds a 90 Hz, amplitude-0.5 sine wave to 20% of spoofed training samples in the ASVSpoof 2019 dataset, targeting a self-built replica of the VA+VoicePop anti-spoofing system. The authors report that the attack reduces classification accuracy to 14.01% with a false-negative rate of 95.88%, compared with 37.52% for the even-train baseline and 37.21% for a label-flipping attack at the same poisoning rate, and they interpret this as exposing a new vulnerability in pop-noise-based voice authentication.

Significance. If the result holds, it identifies a plausible training-time vulnerability in pop-noise-based liveness detection and is one of the few evaluations of VoicePop against logical/poisoning attacks. The paper has several strengths: the attack procedure is given explicitly in Algorithm 2, the comparison against a label-flipping baseline is methodologically useful, and the use of the public ASVSpoof 2019 dataset is a practical improvement over the original 18-participant study. However, the headline metrics are entangled with threshold choice, class imbalance, and an unreleased self-built replica, so the magnitude of the claimed effect is not yet established. The central phenomenon is testable, but the current evidence supports only a qualitative statement that low-frequency pop injection can bias a pop-noise-based detector.

major comments (5)
  1. [Section V-B, Table II] The reported "attack success rate of over 95%" is not computed on the poisoned samples. The paper's own text equates ASR with the false-negative rate of 95.88% over the entire evaluation set, which contains 63,882 spoof and 7,355 real samples. Because an all-real classifier would already achieve roughly 10.3% accuracy on that split, the 14.01% accuracy does not by itself demonstrate that the 20% poisoned training subset caused the failure. Please report ASR separately for poisoned and unpoisoned spoof samples, and include class-balanced metrics such as balanced accuracy and AUC.
  2. [Sections V-A and V-B, Figures 6-7] The decision threshold appears to be chosen after inspecting the test-score distribution: the even-train baseline uses 0.5, the label-flip run uses 0.48, and the SyntheticPop distribution is described as centered around 0.44. This makes the accuracy comparison threshold-dependent, and the TNR=100% row in Table II is consistent with a threshold that trivially accepts all real samples. Please fix a single threshold a priori (for example, from a validation split) and report ROC or precision-recall curves, or demonstrate that the headline numbers are stable over a range of thresholds.
  3. [Section III-B, Table I] The self-built replica of VA+VoicePop is not validated against the original system. The original code and the 18-participant dataset are unavailable, and the replication uses a different codebase, SMOTE, balanced weights, and GridSearch on ASVSpoof 2019. The resulting even-train baseline accuracy is 37.52%, far below the 93.5% replay-detection accuracy reported for VoicePop, so the claims in the paper are about a replica-plus-ASVSpoof system rather than about the original VoicePop. Please provide evidence of feature-level fidelity, evaluate on the original dataset if it can be obtained, or at least evaluate on a replay-only subset and discuss how the low baseline affects the attack conclusions.
  4. [Section IV, Algorithm 2] The attack injects a 90 Hz sine wave at amplitude 0.5, which directly matches the below-100 Hz pop-noise feature that the defense explicitly extracts. No sensitivity analysis is provided for the amplitude A or frequency f, so the attack may succeed only because it is hand-tuned to the detector's known feature. Please include a sensitivity sweep over A and f, and evaluate against a minimally adaptive defense (for example, a notch filter at 90 Hz or a shifted pop-noise frequency band) to show the attack is not trivially invalidated.
  5. [Tables I-II] Each experiment is reported from a single run, and the poisoning experiments randomly select which 20% of samples are modified. Given the random selection and the small accuracy differences between the label-flip attack and the even-train baseline, the reported differences may be within run-to-run variance. Please report results over multiple random seeds with means and standard deviations, or at least provide a confidence interval for the key 14.01% versus 37.21% comparison.
minor comments (6)
  1. [Section II] There is a typo: "SytheticPop" should be "SyntheticPop" in the sentence "deploying our SytheticPop attacks against it."
  2. [Section VI] The word "determent" should be "detriment" in the sentence "This type of attack proved to be a determent to the overall system."
  3. [Algorithm 2, line 5] The time vector expression "t[i] = i * i * D / len(a)" appears to be a typo; if the intended formula is t[i] = i * D / len(a), the current expression would produce an incorrect quadratic time base.
  4. [Section III-B] The sentence about the starter codebase cites reference [5], which is the Mochizuki et al. pop-noise paper, while the GitHub repository appears to be reference [6]; the citation should be corrected.
  5. [Section V-A] The text says the threshold "was slightly lowered by -0.2 compared to the 0.5" threshold; since 0.48 is a decrease of 0.02, the phrase should read "lowered by 0.02."
  6. [Table II] For consistency, the entry "95.88" should be written as "95.88%" in the FNR column.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the attack is deliberately matched to the defense's pop-noise feature, but the reported degradation is an empirical training outcome, not a derived tautology.

full rationale

The paper's central claim is an empirical attack result: poisoning 20% of spoof training samples with a 90 Hz sine wave reduces the replicated VA+VoicePop SVM's accuracy to 14% and raises the false-negative rate to 95.88%. Algorithm 2 defines the poison as a 90 Hz sine wave because VoicePop's STFT stage isolates energy below 100 Hz, so the attack is intentionally aligned with the defense's target feature. However, this is an attack-design choice, not a circular derivation. The paper does not derive the accuracy drop from the sine-wave equation; it measures the drop after retraining. The SVM could in principle have remained robust to the added sine wave, or the poisoned-label-consistent samples could have increased TPR rather than collapsing it; the observed 95.88% FNR is a contingent empirical result of the model's learning dynamics. The label-flipping baseline (37.21% accuracy, roughly unchanged from the even-train baseline) provides an independent point of comparison, and the full-train and even-train baselines give context. No self-citations are load-bearing: references [2], [4], [5], and [6] are external prior work, and the replication uses an external codebase and ASVSpoof 2019. There is a citation error (Section III-B cites [5] for the starter codebase that is actually listed as [6]), but that is a reference-accuracy issue, not circularity. The skeptic concerns about post-hoc threshold selection (0.61 full-train, 0.48 label-flip) and about the attack-success rate being the global false-negative rate rather than a rate on poisoned samples are metric-validity and robustness concerns; they do not make any prediction equivalent to its inputs. Under the hard rule that circularity must be exhibited as a specific reduction or a fitted parameter renamed as a prediction, no such step is present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central attack rests on two hand-tuned signal parameters, per-experiment thresholds tuned on the evaluation set, and unverified assumptions that the replica faithfully matches VoicePop. No new physical entities are introduced; SyntheticPop is a signal perturbation method, not a postulated entity.

free parameters (4)
  • Sine wave amplitude A = 0.5
    Set by hand in Algorithm 2 so the added tone is strong enough to be detected by the GFCC/pop-noise front end; no sensitivity analysis or principled selection.
  • Sine wave frequency f = 90 Hz
    Chosen to sit below the 100 Hz pop-noise band that the defense's STFT stage targets (Section III.B and Algorithm 2), making the attack explicitly matched to the detector.
  • Decision threshold = 0.61 (full train), 0.5 (even train), 0.48 (label flip)
    Selected per experiment based on the evaluation-set probability distribution (Section III.C), which tunes the metric on the test data and may inflate reported accuracy.
  • Poisoning fraction = 20%
    Chosen attack strength; no dose-response curve is reported beyond a note that higher poisoning likely worsens accuracy.
assumptions (4)
  • domain assumption The authors' reimplementation of VA+VoicePop using ASVSpoof2019 and GFCC features is a faithful proxy for the original VoicePop system.
    Original code and data are unavailable, and features were rebuilt from a related codebase (Section III.B). If the replica differs from VoicePop, the attack results do not transfer.
  • domain assumption ASVSpoof 2019 spoof labels can stand in for the replay-attack distribution VoicePop was designed to detect.
    The dataset mixes replay, TTS, and deepfake attacks; the paper acknowledges this in Section VI and cannot isolate replay-only behavior.
  • ad hoc to paper A 90 Hz sine wave added at amplitude 0.5 is a valid surrogate for a genuine pop noise in the STFT/GFCC pipeline.
    Algorithm 2 posits that the synthetic tone will be picked up as pop noise; no acoustic analysis verifies this beyond the observed classifier shift.
  • standard math Standard SVM training with SMOTE and class balancing on these features reproduces the decision behavior of the target classifier.
    The paper relies on scikit-learn-style SVM/GridSearch and five-fold cross-validation as a standard baseline (Section III.B), without reporting final hyperparameters.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SyntheticPop: Attacking Speaker Verification Systems With Synthetic VoicePops." pith.science (2026). https://pith.science/paper/BXJJN5VP

@misc{pith2026250209553,
  author       = {Pith},
  title        = {Pith review of: SyntheticPop: Attacking Speaker Verification Systems With Synthetic VoicePops},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BXJJN5VP}},
  note         = {Machine review of arXiv:2502.09553}
}
read the original abstract

Voice Authentication (VA), also known as Automatic Speaker Verification (ASV), is a widely adopted authentication method, particularly in automated systems like banking services, where it serves as a secondary layer of user authentication. Despite its popularity, VA systems are vulnerable to various attacks, including replay, impersonation, and the emerging threat of deepfake audio that mimics the voice of legitimate users. To mitigate these risks, several defense mechanisms have been proposed. One such solution, Voice Pops, aims to distinguish an individual's unique phoneme pronunciations during the enrollment process. While promising, the effectiveness of VA+VoicePop against a broader range of attacks, particularly logical or adversarial attacks, remains insufficiently explored. We propose a novel attack method, which we refer to as SyntheticPop, designed to target the phoneme recognition capabilities of the VA+VoicePop system. The SyntheticPop attack involves embedding synthetic "pop" noises into spoofed audio samples, significantly degrading the model's performance. We achieve an attack success rate of over 95% while poisoning 20% of the training dataset. Our experiments demonstrate that VA+VoicePop achieves 69% accuracy under normal conditions, 37% accuracy when subjected to a baseline label flipping attack, and just 14% accuracy under our proposed SyntheticPop attack, emphasizing the effectiveness of our method.

Figures

Figures reproduced from arXiv: 2502.09553 by the authors.

Figure 1
Figure 1. Overview of the VA+VoicePop architecture by Wang et al. [2] [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Probability distribution curve for real samples in Full Training mode [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Probability distribution curve for real samples in Even Training mode [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Top: Unpoisoned spoofed audio file, Bottom: The same file after [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: Probability distribution curve for real samples 20% Poison with Label [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 3
Figure 3. Figure 3: It should be noted that the threshold was slightly [PITH_FULL_IMAGE:figures/full_fig_p005_3.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

16 extracted references · 10 canonical work pages

  1. [1]

    J. Zhou, T. Hai, D. N. A. Jawawi, D. Wang, E. Ibeke, and C. Bi- amba, ”V oice spoofing countermeasure for voice replay attacks using deep learning,” Journal of Cloud Computing: Advances, Systems, and Applications, vol. 11, no. 51, pp. 1–14, 2022. [Online]. Available: https://doi.org/10.1186/s13677-022-00306-5

    work page doi:10.1186/s13677-022-00306-5 2022

  2. [2]

    Q. Wang, X. Lin, M. Zhou, Y . Chen, C. Wang, Q. Li, and X. Luo, ”V oicePop: A pop noise-based anti-spoofing system for voice authentication on smartphones,” in Proceedings of the IEEE Interna- tional Conference on Communications (ICC) , 2024. [Online]. Available: https://doi.org/10.48550/arXiv.2007.08199

    arXiv doi:10.48550/arxiv.2007.08199 2024

  3. [3]

    Valero and F

    X. Valero and F. Al ´ıas, ”Gammatone cepstral coefficients: Biologically inspired features for non-speech audio classification,” IEEE Transactions on Multimedia , vol. 14, no. 6, pp. 1684–1692, Dec. 2012. [Online]. Available: https://doi.org/10.1109/TMM.2012.2199972

    work page arXiv 2012

  4. [4]

    Yamagishi, M

    J. Yamagishi, M. Todisco, M. Sahidullah, H. Delgado, X. Wang, N. Evans, T. Kinnunen, K. A. Lee, V . Vestman, and A. Nautsch, ”ASVspoof 2019: The 3rd Automatic Speaker Verification Spoofing and Counter- measures Challenge database,” University of Edinburgh, The Centre for Speech Technology Research (CSTR) , 2019. [Online]. Available: https://doi.org/10.748...

    work page doi:10.7488/ds/2555 2019

  5. [5]

    Mochizuki, S

    S. Mochizuki, S. Shiota, and H. Kiya, ”V oice liveness detection based on pop-noise detector with phoneme information for speaker verification,” Journal of the Acoustical Society of America , vol. 140, no. 4, pp. 3060,

    work page

  6. [6]

    Merchant, V oice Authentication and Face Recognition, GitHub repos- itory, 2024

    M. Merchant, V oice Authentication and Face Recognition, GitHub repos- itory, 2024. [Online]. Available: https://github.com/MohamadMerchant/ V oice-Authentication-and-Face-Recognition

    work page 2024

  7. [7]

    Q. Xu, Z. Yang, Y . Zhao, X. Cao and Q. Huang, ”Rethinking Label Flipping Attack: From Sample Masking to Sample Thresholding,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 6, pp. 7668-7685, June 2023. [Online] Available: https://doi.org/ 10.1109/TPAMI.2022.3220849

    work page arXiv 2023

  8. [9]

    H. Song, M. Kim, D. Park, J. Shin, and J.-G. Lee, ”Learning from noisy labels with deep neural networks: A survey,” ResearchGate, Mar. 2022. Available: https://doi.org/10.1109/TNNLS.2022.3152527

    arXiv 2022

Show all 16 references
  1. [10]

    Mittal and M

    A. Mittal and M. Dua, ”Automatic speaker verification systems and spoof detection techniques: review and analysis,” International Journal of Speech Technology , Aug. 2021. [Online]. Available: https://doi.org/ 10.1007/s10772-021-09876-2

    2021 doi

  2. [11]

    Kinnunen, M

    T. Kinnunen, M. Sahidullah, H. Delgado, M. Todisco, N. Evans, J. Yamagishi, and K. A. Lee, ”The ASVspoof 2017 Challenge: Assessing the Limits of Replay Spoofing Attack Detection,” Interspeech 2017 , Aug. 2017. [Online]. Available: https://doi.org/10.21437/Interspeech. 2017-1111

    2017 doi

  3. [12]

    G. Chen, S. Chen, L. Fan, X. Du, Z. Zhao, F. Song, and Y . Liu, ”Who is Real Bob? Adversarial Attacks on Speaker Recognition Systems,” arXiv preprint, Apr. 2020. [Online]. Available: https://doi.org/10.48550/arXiv. 1911.01840

    2020 doi arXiv

  4. [13]

    Zhang, C

    C. Zhang, C. Zhang, S. Zheng, M. Zhang, M. Qamar, S.-H. Bae, and I. S. Kweon, ”A Survey on Audio Diffusion Models: Text-To-Speech Synthesis and Enhancement in Generative AI,” arXiv preprint , Apr

  5. [14]

    SMOTE: synthetic minority over-sampling technique,

    N. V . Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: synthetic minority over-sampling technique,” Journal of Ar- tificial Intelligence Research, vol. 16, no. 1, pp. 321–357, Jun. 2002. [Online]. Available: https://doi.org/10.5555/1622407.1622416

    2002 arXiv

  6. [2016]

    Available: https://doi.org/10.1121/1.4969520

    [Online]. Available: https://doi.org/10.1121/1.4969520

    doi

  7. [2022]

    Available: https://arxiv.org/abs/2202.10276

    [Online]. Available: https://arxiv.org/abs/2202.10276

    arXiv

  8. [2023]

    Available: https://doi.org/10.48550/arXiv.2303.13336

    [Online]. Available: https://doi.org/10.48550/arXiv.2303.13336

    doi arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.