Pith. sign in

REVIEW 5 major objections 5 minor 57 references

Fake speech breaks the natural low–high frequency link in real voices, and a detector built around that link beats prior systems.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-03 20:00 UTC pith:2HD7ZFQ2

load-bearing objection Promising frequency-contrastive audio deepfake detector with a real test-set selection problem and a wrong theory. the 5 major comments →

arxiv 2511.21325 v2 pith:2HD7ZFQ2 submitted 2025-11-26 cs.SD cs.AI

SONAR: Spectral-Contrastive Audio Residuals for Generalizable Deepfake Detection

Ido Nitzan Hidekel , Gal lifshitz , Khen Cohen , Dan Raviv This is my paper
classification cs.SD cs.AI
keywords audio deepfake detectionspectral biasJensen-Shannon divergencehigh-frequency residualslearnable SRM filterscontrastive representation learningout-of-distribution generalizationspeech anti-spoofing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that synthetic speech leaves a detectable signature in the joint statistics of low- and high-frequency content: real voices co-modulate across bands (correlation near 0.6), while fakes drift toward independence or negative correlation. Based on that, SONAR splits audio into a content stream and a noise stream, uses a learnable high-pass filter bank to isolate faint high-frequency residuals, and applies a Jensen–Shannon divergence loss that pulls the two streams together for real audio and pushes them apart for fakes. The authors report state-of-the-art equal-error rates on the ASVspoof 2021 and In-the-Wild benchmarks, along with roughly 4–8× faster convergence than a strong baseline. The broader point is that spectral bias—networks' habit of learning low frequencies first—can be inverted into a supervisory signal.

Core claim

SONAR's claim is that the failure of deepfake generators to reproduce the natural dependency between a speech signal's low-frequency content and its high-frequency microstructure is a reliable, learnable cue. The detector splits each utterance into content and noise paths with a bank of constrained high-pass filters, encodes both with the same pre-trained speech encoder, and regulates the Jensen–Shannon divergence between their softmaxed embeddings: minimization for genuine audio, maximization for fakes. Cross-attention fuses the two views before classification. The paper reports equal-error rates of 1.57% on the deepfake subset, 1.55% on logical access, and 6.00% on in-the-wild audio in 12

What carries the argument

The load-bearing mechanism is the pair formed by a learnable constrained high-pass filter bank—each 5-tap filter forced to have center coefficient −1 and zero sum, so it suppresses content and passes residuals—and a frame-wise Jensen–Shannon divergence loss that treats softmaxed embeddings as probability distributions. The loss actively aligns low- and high-frequency embeddings for real speech and separates them for fakes, converting the generator's high-frequency 'hole' into a margin that a cross-attention fusion then feeds to the classifier.

Load-bearing premise

The method rests on the bet that real speech always couples low- and high-frequency content while fake audio does not; if a future voice generator restores that coupling, the alignment loss would pull the wrong pairs together.

What would settle it

Measure the frame-wise low–high frequency correlation (the paper's Fig. 2 statistic) on a fresh, held-out set of modern TTS and vocoder outputs; if fake samples show a correlation near 0.6, matching real speech, and the detector still flags them through other cues, then the coupling assumption is not the operative one. Alternatively, train a generator with a loss that explicitly enforces low–high frequency co-modulation; if SONAR's error rate does not rise toward chance, the alignment loss is not doing the claimed work.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the coupling signal is real, detectors need not wait for deep classifiers to stumble on high-frequency artifacts; explicitly aligning low/high-frequency embeddings yields state-of-the-art out-of-distribution error rates.
  • Training stabilizes in 12 epochs (and 4–6 for the fine-tuned variant) because the alignment loss gives the optimizer a clearer gradient than classification alone.
  • Because the loss operates on embeddings, it can be added to any audio backbone without retraining from scratch, and the authors argue the same representation-level principle transfers to images or video.
  • Robustness to common codecs holds, though aggressive downsampling that removes high frequencies degrades detection—a limitation the paper states.
  • The learned high-pass filters are data-driven rather than hand-crafted, so the frequency split adapts to whatever artifacts the training data expose.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct extension: measure the frame-wise low–high frequency correlation on newly released TTS systems; if newer generators restore co-modulation, SONAR's margin should shrink, suggesting the cue is being consumed rather than permanent.
  • The same co-modulation principle may transfer beyond audio: image and video deepfakes also exhibit spectral bias, and a two-stream aligned residual scheme could be ported with modality-specific filters.
  • The convergence speed-up hints that the alignment term acts like privileged information or a curriculum, making the loss potentially useful in semi-supervised or self-supervised pre-training for forgery detection.
  • An adversarial angle: a generator trained with a joint spectrogram loss that explicitly enforces low–high frequency coupling could evade the detector, so robustness evaluations should include such coupling-preserving attacks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes SONAR, a dual-path audio deepfake detector that pairs a content encoder (XLSR) on the raw waveform with a noise encoder (XLSR) applied to the output of a learnable, value-constrained high-pass SRM filter bank. The content and noise embeddings are fused by cross-attention and classified by AASIST. A Jensen–Shannon divergence alignment loss pulls the content and noise embeddings together for real audio and pushes them apart for fake audio (Eqs. 1 and 8). The authors report state-of-the-art equal error rates on ASVspoof 2021 LA/DF and In-the-Wild, with faster convergence than the XLSR+AASIST baseline, and support this with ablations, t-SNE/embedding analyses, and robustness checks. The paper also claims a theoretical justification based on a Pinsker-style bound on Bayes error.

Significance. If the empirical claims hold, the paper offers a practical and conceptually interesting contribution: explicitly routing high-frequency residuals through a separate, learnable branch and aligning low- and high-frequency embeddings improves both OOD generalization and convergence speed for audio deepfake detection. The ablations indicate that both the RFE module and the JS alignment loss contribute to the gains, and the three-seed runs with significance tests are a step beyond the field's usual single-run reporting. However, the theoretical motivation contains a mathematically incorrect bound, and the reported SOTA numbers appear to be obtained by selecting hyperparameters on the test sets, which undermines the reliability of the central claim. The paper is therefore not yet ready for acceptance; the empirical results are promising but need to be presented with a correct theory and a proper model-selection protocol.

major comments (5)
  1. [Sec. 3, 'Error bound'] The claimed Pinsker bound 'P_e ≤ 1/2 sqrt(2 D_JS)' is not a valid upper bound on Bayes error. For D_JS = 0 the right-hand side is 0, while the Bayes error for identical distributions is 1/2. The correct relationship (from Pinsker-type inequalities) gives a lower bound on P_e in terms of D_JS, not an upper bound that shrinks to 0. Consequently, the statement that Eq. (1) 'shrinks P_e for genuine pairs' is not justified. This is load-bearing for the paper's mathematical motivation; the authors should either correct the inequality or explicitly reframe the alignment loss as a heuristic without the erroneous bound.
  2. [Sec. 5.3 / Table 2 and Sec. 4.2] The hyperparameters M (number of SRM filters) and λ_JS are selected by comparing EERs on the held-out DF, LA, and ITW test sets in Table 2. For example, λ_JS=0.8 yields ITW 7.02% versus 6.00% for λ_JS=1, and M=10 yields 7.40% versus 6.00% for M=30; the configuration presented as best in Table 1 is the one with the lowest test EER. This is a selection-on-test-set bias that inflates the reported SOTA and makes comparisons with baselines unfair. The paper should report a proper protocol: choose M and λ_JS on the ASVspoof 2019 LA validation set, then evaluate on the test sets. Without this, the central SOTA claim is not reliable.
  3. [Sec. 4.3, Table 1] The text states that 'SONAR-Lite still attains state-of-the-art performance,' but Table 1 contradicts this: SONAR-Lite is worse than XLSR-Mamba on all three benchmarks (LA 1.78 vs 0.93, DF 2.11 vs 1.88, ITW 6.98 vs 6.71). This is a factual error in the description of the model's performance and should be corrected.
  4. [Sec. 5.2, LA discussion] The paper explains the LA gap vs. XLSR-Mamba by claiming that baseline results use checkpoint averaging or run-smoothing, while SONAR is evaluated under a 'fair single-run protocol,' and then concludes that SONAR achieves SOTA on LA. No single-run baseline numbers are provided for XLSR-Mamba or other baselines, so the reader cannot verify the claim. The authors should either include the single-run results for the baselines under the same protocol or temper the 'single-run SOTA on LA' claim.
  5. [Sec. 5.3, Fig. 4 and Eq. (8)] The embedding analysis in Fig. 4 is presented as evidence that SONAR 'yields a more discriminative latent space,' but the real/fake separation in cosine similarity is largely a direct consequence of the JS alignment loss in Eq. (8), which explicitly maximizes the divergence between content and noise embeddings for fake samples. This is a sanity check of the training objective rather than an emergent or independent finding. The paper should frame it as such and avoid over-interpreting it as evidence of a 'disjoint manifold' property beyond what the loss enforces.
minor comments (5)
  1. [Introduction, refs [1-3]] The sentence 'voice-cloning scams caused multimillion-dollar losses, including a 25M$ transfer' cites [1,2], but the specific $25M incident is reference [3]; [1] and [2] do not appear to contain that number.
  2. [Appendix, Fig. 5] The caption and surrounding text are garbled: 'The degradation from a state-of-the-art 6 demonstrates...' appears to be missing a number or a phrase. Please fix.
  3. [Sec. 5.1 / Table 2] Table 2's robustness rows use '≈0', 'medium jitter', and 'small jitter' without a clear definition or units; the paper should specify how these are measured and how they relate to EER or probability shifts.
  4. [Abstract / Conclusion] The abstract says SONAR converges 'four times faster,' while the conclusion says 'up to 8× faster.' These numbers should be reconciled, and the comparison basis (baseline training epochs) should be stated explicitly.
  5. [Table 1 significance tests] The t-stats (e.g., t=19.4, p=0.0026) are reported without degrees of freedom or variance information, and with only 3 runs for SONAR and a single run for the baseline, the meaning of the t-test is unclear. Please report the full statistical details or use a more appropriate test.

Circularity Check

2 steps flagged

Embedding 'disjoint manifolds' restates the JS training objective; M and λ_JS are selected from test-set ablations, inflating SOTA.

specific steps
  1. self definitional [Sec. 4.2.1 Eq. 8; Fig. 4; Abstract/Conclusion]
    "LJS(x, y) = y·JS(z_c,z_n) + (1−y)·(1−JS(z_c,z_n)) (8) ... Cosine similarity histograms confirm that real speech preserves LF–HF coupling, while fakes exhibit disjoint embeddings."

    Eq. 8 is exactly the training objective: for real samples (y=1) it minimizes the JS divergence between content and noise embeddings, and for fakes (y=0) it maximizes that divergence. Fig. 4's displayed pattern — real cosine similarity near 1, fake near −0.2 — is therefore the optimized quantity itself, not an emergent property. The abstract/conclusion claim that SONAR 'splits the latent space into two disjoint manifolds' restates what the loss was constructed to do. Thus the embedding analysis is a sanity check of the fitted objective, not an independent prediction or validation of the mechanism.

  2. fitted input called prediction [Sec. 4.2.1; Sec. 5.3, Table 2; Sec. 5.1]
    "After ablation study, we chose to be λJS =1. ... Top: pooled EER (%) on DF, LA, and ITW sets under different architectural ablations. ... SONAR-Full w/ SRM and λ_JS=1(best configuration) 1.57 1.55 6.00 ... Varying M and λ_JS shows consistent robustness, with λ_JS=1 yielding the strongest overall results in the ablation."

    Table 2 reports EER on the held-out DF, LA, and ITW test sets for every value of M and λ_JS, and the paper selects M=30, λ_JS=1 because it has the lowest test EER. Those same selected test-set numbers are then presented in Table 1 as the SOTA result. This is hyperparameter selection directly on the evaluation metric: the reported 'prediction' is the best of many test-set evaluations, so it is statistically forced upward and is not an independent estimate. Sec. 5.1 states that only the ASVspoof 2019 LA validation set was used for tuning, which the test-set ablation procedure contradicts. A fair protocol would choose M and λ_JS on the validation set and evaluate on the test sets once.

full rationale

The paper's core architecture — dual-path XLSR with constrained SRM high-pass filters, cross-attention fusion, and the JS alignment loss — is not circular in itself: it is implemented and evaluated against external ASVspoof 2021 and In-the-Wild benchmarks, with published baselines, so the EER comparisons carry independent content. However, two load-bearing steps reduce to their own inputs. First, the 'disjoint manifolds' / LF-HF coupling finding (Fig. 4, abstract, conclusion) is literally the training objective of Eq. 8: minimize JS divergence for real, maximize for fake; displaying cosine similarities near 1 versus −0.2 is reporting the fitted loss, not an emergent discovery. Second, Table 2 selects M=30 and λ_JS=1 by comparing EERs on the DF/LA/ITW test sets, and those selected test-set numbers are then presented as SOTA, which is selection-on-test-set bias and makes the headline numbers partially fitted to the benchmark. No load-bearing self-citation chain is present; the Fig. 2 factorization motivation is descriptive, not circular, though it is computed on the same corpora used for training/testing. Overall this is partial circularity: the latent-separation claim reduces by construction, and the SOTA claim is inflated by test-set hyperparameter selection.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 0 invented entities

The method rests on tuned hyperparameters M and λ_JS and on the spectral-bias/factorization assumptions. The factorization assumption is the most paper-specific; its evidence is descriptive statistics computed on the same datasets later used for evaluation, not an independent test.

free parameters (2)
  • M (number of SRM filters) = 30
    Selected via ablation on test sets (Table 2: M=1,10,30); controls NFE capacity and is central to the high-pass residual extraction.
  • λ_JS (alignment loss weight) = 1
    Chosen after ablation on test sets (Table 2: λ_JS=0.5, 0.8, 1); balances weighted cross-entropy and the JS loss in Eq. 9.
axioms (5)
  • domain assumption Deep networks exhibit spectral bias (frequency principle), learning low frequencies first
    Invoked in Secs. 1 and 3 to justify concentrating on high-frequency residuals; well-supported by cited literature [7-11] but not derived here.
  • ad hoc to paper Real speech satisfies preal(L,H) ≠ p(L)p(H); fake speech approximately factorizes pfake(L,H) ≈ p(L)p(H)
    Stated in Sec. 3; the only evidence is Fig. 2's correlation/energy plots on the same corpora used for training and evaluation, which do not establish factorization.
  • standard math Pinsker inequality gives the Bayes error bound P_e ≤ 1/2 sqrt(2 D_JS)
    Invoked in Sec. 3 'Error bound'; the inequality is misapplied as an upper bound on Bayes error (direction fails for small D_JS), so the stated bound is invalid.
  • domain assumption XLSR content embeddings are low-frequency dominated
    Assumed in Sec. 4.1; the paper does not measure or cite analysis of XLSR's frequency selectivity, making the LF/HF split an unverified modeling choice.
  • domain assumption High-frequency artifacts persist in synthetic audio after codecs/resampling
    Needed for the NFE branch to be useful; the paper's own Limitations section shows sensitivity to downsampling, weakening this for low-bandwidth inputs.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of SONAR: Spectral-Contrastive Audio Residuals for Generalizable Deepfake Detection." pith.science (2026). https://pith.science/paper/2HD7ZFQ2

@misc{pith2026251121325,
  author       = {Pith},
  title        = {Pith review of: SONAR: Spectral-Contrastive Audio Residuals for Generalizable Deepfake Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2HD7ZFQ2}},
  note         = {Machine review of arXiv:2511.21325}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Deepfake (DF) audio detectors still struggle to generalize to out of distribution inputs. A central reason is spectral bias, the tendency of neural networks to learn low-frequency structure before high-frequency (HF) details, which both causes DF generators to leave HF artifacts and leaves those same artifacts under-exploited by common detectors. To address this gap, we propose Spectral-cONtrastive Audio Residuals (SONAR), a frequency-guided framework that explicitly disentangles an audio signal into complementary representations. An XLSR encoder captures the dominant low-frequency content, while the same cloned path, preceded by learnable SRM, value-constrained high-pass filters, distills faint HF residuals. Frequency cross-attention reunites the two views for long- and short-range frequency dependencies, and a frequency-aware Jensen-Shannon contrastive loss pulls real content-noise pairs together while pushing fake embeddings apart, accelerating optimization and sharpening decision boundaries. Evaluated on the ASVspoof 2021 and in-the-wild benchmarks, SONAR attains state-of-the-art performance and converges four times faster than strong baselines. By elevating faint high-frequency residuals to first-class learning signals, SONAR unveils a fully data-driven, frequency-guided contrastive framework that splits the latent space into two disjoint manifolds: natural-HF for genuine audio and distorted-HF for synthetic audio, thereby sharpening decision boundaries. Because the scheme operates purely at the representation level, it is architecture-agnostic and, in future work, can be seamlessly integrated into any model or modality where subtle high-frequency cues are decisive.

Figures

Figures reproduced from arXiv: 2511.21325 by Dan Raviv, Gal Lifshitz, Ido Nitzan Hidekel, Khen Cohen.

Figure 1
Figure 1. Figure 1: SONAR overview. Audio is processed in parallel by the Content Feature Extractor (CFE) and the Noise Feature Extractor (NFE). Their embeddings are fused via cross-attention (CA) and classified as real/fake. et al. [33] similarly boost noise for image forgery detection, treating it as an auxiliary cue that still requires pixel-level masks. None of these approaches capture the higher-order dependency between … view at source ↗
Figure 2
Figure 2. Figure 2: Low–high frequency structure reveals spoofing artifacts. (a) Pearson correlation be￾tween low- (0–4 kHz) and high-frequency (7–8 kHz) bands shows real speech with strong co￾modulation (r ≈ 0.6), while fakes collapse toward zero or negative values. (b) The energy difference ∆E = EHF − ELF is systematically shifted for fakes across corpora, exposing a consistent HF/LF imbalance. These second-order cues motiv… view at source ↗
Figure 3
Figure 3. Figure 3: Rich Feature Extractor (RFE). Audio x is processed by a bank of M SRM-inspired filters, concatenated, and passed through a 1 × 1 learnable convolution layer to produce the noise residual representation xnoise. Dual-path embeddings. We split X with an ideal band-pass filter, L = FlowX, H = FhighX, feed each band to the same encoder ϕθ to obtain zcontent = ϕθ(L), znoise = ϕ HF θ [PITH_FULL_IMAGE:figures/ful… view at source ↗
Figure 4
Figure 4. Figure 4: Latent representation analysis of SONAR. (a) t-SNE shows that SONAR’s dual-path embeddings separate real and fake audio more distinctly than the baseline. (b) Cosine similarity histograms confirm that real speech preserves LF–HF coupling, while fakes exhibit disjoint embed￾dings. 6 CONCLUSION SONAR reframes audio–deepfake detection as a frequency-guided, contrastive representation task. By splitting speech… view at source ↗
Figure 5
Figure 5. Figure 5: Impact of resampling on detection accuracy. Equal-Error Rate (EER) rises as the sampling [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Inference latency scales linearly with audio length. We compare inference times (in seconds) for the XLSR-Mamba and SONAR-Full models across increasing audio durations from 1 to 10 seconds. SONAR introduces only a minimal overhead relative to XLSR-Mamba, while delivering improved detection performance (cf [PITH_FULL_IMAGE:figures/full_fig_p017_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 9 linked inside Pith

  1. [1]

    Super Year

    United Nations Development Programme. A “Super Year” for Elections – Strengthening Democracy and Good Governance in 2024. 2024. Available athttps://www.undp.org/ blog/super-year-elections-2024. Accessed: 2025-07-27

    2024

  2. [2]

    AI-Enabled Fraud: How Scammers Are Exploiting Gener- ative AI

    TRM Labs. AI-Enabled Fraud: How Scammers Are Exploiting Gener- ative AI. 2025. Available athttps://www.trmlabs.com/post/ ai-enabled-fraud-how-scammers-are-exploiting-generative-ai. Accessed: 2025-07-27

    2025

  3. [3]

    G. Noto. Scammers Siphon $25M from Engineering Firm Arup via AI Deep- fake “CFO”. 2024. Available athttps://www.cfodive.com/news/ arup-scammer-deepfake-cfo/715302/. Accessed: 2025-07-27

    2024

  4. [4]

    Tan et al

    C. Tan et al. Frequency-Aware Deepfake Detection: Improving Generalizability Through Fre- quency Space Domain Learning. InProceedings of the AAAI Conference on Artificial Intelli- gence, 38(5):5976–5984, 2024. AAAI Press

    2024

  5. [5]

    Z. K. Abdul and A. K. Al-Talabani. Mel Frequency Cepstral Coefficient and Its Applications: A Review.IEEE Access, 10:122136–122158, 2022

    2022

  6. [6]

    FBI Warns Senior US Officials Are Being Impersonated Using Texts, AI V oice Messages

    Cybersecurity Dive. FBI Warns Senior US Officials Are Being Impersonated Using Texts, AI V oice Messages. 2025. Available athttps://www.cybersecuritydive. com/news/fbi-warns-senior-us-officials-being-impersonated-ai/ 716824/. Accessed: 2025-07-27

    2025

  7. [7]

    Rahaman, A

    N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y . Bengio, and A. Courville. On the Spectral Bias of Neural Networks. InProceedings of the 36th Inter- national Conference on Machine Learning (ICML), volume 97, pages 5301–5310. PMLR, 2019

    2019

  8. [8]

    Basri, D

    R. Basri, D. Jacobs, Y . Kasten, and S. Kritchman. The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies. InAdvances in Neural Information Processing Systems 32 (NeurIPS), pages 4761–4771. Curran Associates, Inc., 2019

    2019

  9. [9]

    Y . Cao, Z. Fang, Y . Wu, D.-X. Zhou, and Q. Gu. Towards Understanding the Spectral Bias of Deep Learning. arXiv:1912.01198, 2019

    Pith/arXiv arXiv 1912

  10. [10]

    Z.-Q. J. Xu, Y . Zhang, and T. Luo. Overview Frequency Principle/Spectral Bias in Deep Learning.Communications on Applied Mathematics and Computation, pages 1–38, 2024. Early access

    2024

  11. [11]

    Fridovich-Keil, R

    S. Fridovich-Keil, R. G. Lopes, and R. Roelofs. Spectral Bias in Practice: The Role of Func- tion Frequency in Generalization. InAdvances in Neural Information Processing Systems 35 (NeurIPS), pages 7368–7382, 2022

    2022

  12. [12]

    Shwartz-Ziv and N

    R. Shwartz-Ziv and N. Tishby. Opening the Black Box of Deep Neural Networks via Informa- tion. arXiv:1703.00810, 2017

    Pith/arXiv arXiv 2017

  13. [13]

    Li and Y

    Y . Li and Y . Yuan. Convergence Analysis of Two-layer Neural Networks with ReLU Acti- vation. InAdvances in Neural Information Processing Systems (NeurIPS), pages 597–607, 2017

    2017

  14. [14]

    Allen-Zhu, Y

    Z. Allen-Zhu, Y . Li, and Z. Song. A Convergence Theory for Deep Learning via Over- parameterization. InProceedings of the 36th International Conference on Machine Learning (ICML), volume 97, pages 242–252. PMLR, 2019

    2019

  15. [15]

    Monga, Y

    V . Monga, Y . Li, and Y . C. Eldar. Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing.IEEE Signal Processing Magazine, 38(2):18–44, 2021

    2021

  16. [16]

    Shlezinger, Y

    N. Shlezinger, Y . C. Eldar, M. Goldenbaum, P. Schniter, Y . Murin, D. Gesbert, and G. Leus. Model-based Deep Learning.Proceedings of the IEEE, 111(5):465–499, 2023. 10

    2023

  17. [17]

    Tancik, P

    M. Tancik, P. P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ra- mamoorthi, J. T. Barron, and R. Ng. Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains. InAdvances in Neural Information Processing Sys- tems 33 (NeurIPS), pages 7537–7547, 2020

    2020

  18. [18]

    Y . Qian, G. Yin, L. Sheng, Z. Chen, and J. Shao. Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues. InComputer Vision – ECCV 2020, volume 12367 ofLNCS, pages 86–103. Springer, 2020

    2020

  19. [19]

    M. Yang, Y . Ren, Z. Tang, Z. Lin, X. Qie, and Z. Zhao. WaveGAN: Frequency-aware GAN for High-fidelity Few-shot Image Generation. InComputer Vision – ECCV 2022, volume 13677 ofLNCS, pages 567–584. Springer, 2022

    2022

  20. [20]

    Schwarz, Y

    K. Schwarz, Y . Liao, and A. Geiger. On the Frequency Bias of Generative Models. InAdvances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  21. [21]

    Fuglede and F

    B. Fuglede and F. Topsøe. Jensen–Shannon Divergence and Hilbert Space Embedding. InPro- ceedings of the 2004 IEEE International Symposium on Information Theory (ISIT), page 31. IEEE, 2004

    2004

  22. [22]

    Maltby, J

    H. Maltby, J. Wall, C. Glackin, M. Moniri, N. Cannings, and I. Salami. A Frequency Bin Analysis of Distinctive Ranges Between Human and Deepfake Generated V oices. arXiv:2403.01766, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    W. Yang, J. Feng, J. Yang, F. Zhao, J. Liu, Z. Guo, and S. Yan. Deep Edge Guided Recur- rent Residual Learning for Image Super-resolution.IEEE Transactions on Image Processing, 26(12):5895–5907, 2017

    2017

  24. [24]

    S. Liu, W. Ding, C. Liu, Y . Liu, Y . Wang, and H. Li. ERN: Edge Loss Reinforced Semantic Segmentation Network for Remote Sensing Images.Remote Sensing, 10(9):1339, 2018

    2018

  25. [25]

    I. Masi, A. Killekar, R. M. Mascarenhas, S. P. Gurudatt, and W. AbdAlmageed. Two-Branch Recurrent Network for Isolating Deepfakes in Videos. InEuropean Conference on Computer Vision (ECCV), pages 667–684. Springer, 2020

    2020

  26. [26]

    F. Chollet. Xception: Deep Learning with Depthwise Separable Convolutions. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1251–

  27. [27]

    Jeong, M.-G

    Y .-J. Jeong, M.-G. Kim, H.-U. Jang, H.-T. Lim, and H.-J. Kim. BIHPF: Bilateral High-pass Filters for Robust Deepfake Detection. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 5049–5058. IEEE, 2022

    2022

  28. [28]

    S. Woo. ADD: Frequency Attention and Multi-view Based Knowledge Distillation to De- tect Low-quality Compressed Deepfake Images. InProceedings of the AAAI Conference on Artificial Intelligence, 36(1):1153–1160, 2022

    2022

  29. [29]

    Hertz, O

    A. Hertz, O. Perel, R. Giryes, O. Sorkine-Hornung, and D. Cohen-Or. SAPE: Spatially- adaptive Progressive Encoding for Neural Optimization. InAdvances in Neural Information Processing Systems, volume 34, pages 8820–8832, 2021

    2021

  30. [30]

    Fridrich and J

    J. Fridrich and J. Kodovsk ´y. Rich Models for Steganalysis of Digital Images.IEEE Transac- tions on Information Forensics and Security, 7(3):868–882, 2012

    2012

  31. [31]

    P. Zhou, X. Han, V . I. Morariu, and L. S. Davis. Learning Rich Features for Image Manip- ulation Detection. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1053–1061. IEEE, 2018

    2018

  32. [32]

    Bayar and M

    B. Bayar and M. C. Stamm. A Deep Learning Approach to Universal Image Manipulation Detection Using a New Convolutional Layer. InProceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security, pages 5–10. ACM, 2016. 11

    2016

  33. [33]

    J. Zhu, C. Wang, Y . Zhao, and C. Shen. Learning Discriminative Noise Guidance for Image Forgery Detection and Localization. InProceedings of the AAAI Conference on Artificial Intelligence, 38(7):7913–7921, 2024

    2024

  34. [34]

    Truong, R

    D.-T. Truong, R. Tao, T. Nguyen, H.-T. Luong, K. A. Lee, and E. S. Chng. Temporal-Channel Modeling in Multi-head Self-Attention for Synthetic Speech Detection. InProceedings of Interspeech 2024, pages 537–541, 2024

    2024

  35. [35]

    Xiao and R

    Y . Xiao and R. K. Das. XLSR-Mamba: A Dual-column Bidirectional State Space Model for Spoofing Attack Detection.IEEE Signal Processing Letters, 31:1045–1049, 2024

    2024

  36. [36]

    Y . Guo, H. Huang, X. Chen, H. Zhao, and Y . Wang. Audio Deepfake Detection with Self- supervised WavLM and Multi-fusion Attentive Classifier. InProceedings of the IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 12702–12706. IEEE, 2024

    2024

  37. [37]

    Wang and J

    X. Wang and J. Yamagishi. Investigating Self-supervised Front Ends for Speech Spoofing Countermeasures. InProceedings of Odyssey 2022 – The Speaker and Language Recognition Workshop, pages 100–106. ISCA, 2022

    2022

  38. [38]

    Y . Yang, H. Qin, H. Zhou, C. Wang, T. Guo, K. Han, and Y . Wang. A Robust Audio Deepfake Detection System via Multi-view Feature. InProceedings of the IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP), pages 13131–13135. IEEE, 2024

    2024

  39. [39]

    Pianese, D

    A. Pianese, D. Cozzolino, G. Poggi, and L. Verdoliva. Deepfake Audio Detection by Speaker Verification. InProceedings of the IEEE International Workshop on Information Forensics and Security (WIFS), pages 1–6. IEEE, 2022

    2022

  40. [40]

    Delgado, N

    H. Delgado, N. Evans, T. Kinnunen, K. A. Lee, X. Liu, A. Nautsch, J. Patino, M. Sahidullah, M. Todisco, X. Wang, and J. Yamagishi. ASVspoof 2021: Automatic Speaker Verification Spoofing and Countermeasures Challenge Evaluation Plan. arXiv:2109.00535, 2021

    Pith/arXiv arXiv 2021

  41. [41]

    Z. Wang, R. Fu, Z. Wen, J. Tao, X. Wang, Y . Xie, X. Qi, S. Shi, Y . Lu, Y . Liu, C. Li, X. Liu, and G. Li. Mixture of Experts Fusion for Fake Audio Detection Using Frozen wav2vec 2.0. arXiv:2409.11909, 2024

    Pith/arXiv arXiv 2024

  42. [42]

    J.-W. Jung, H. Tak, V . M. Patel, J. Yamagishi, and N. Evans. AASIST: Audio Anti-spoofing Using Integrated Spectro-temporal Graph Attention Networks. InProceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6369–

  43. [43]

    A. Babu, C. Wang, A. Tjandra, K. Lakhotia, Q. Xu, N. Goyal, K. Singh, P. von Platen, Y . Saraf, J. Pino, A. Baevski, A. Conneau, and M. Auli. XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale. arXiv:2111.09296, 2021

    Pith/arXiv arXiv 2021

  44. [44]

    S. Chen, Y . Cheng, Z. Wang, Y . Qian, J. Wu, D. Yu, and S. Liu. WavLM: Large-scale Self- supervised Pre-training for Full Stack Speech Processing.IEEE Journal of Selected Topics in Signal Processing, 16(6):1505–1518, 2022

    2022

  45. [45]

    Radford, J

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever. Robust Speech Recognition via Large-scale Weak Supervision. arXiv:2212.04356, 2022

    Pith/arXiv arXiv 2022

  46. [46]

    Baevski, H

    A. Baevski, H. Zhou, A. Mohamed, and M. Auli. wav2vec 2.0: A Framework for Self- supervised Learning of Speech Representations. InAdvances in Neural Information Processing Systems 33 (NeurIPS), pages 12449–12460, 2020

    2020

  47. [47]

    Yamagishi, M

    J. Yamagishi, M. Todisco, M. Sahidullah, H. Delgado, X. Wang, N. Evans, T. Kinnunen, K. A. Lee, V . Vestman, and A. Nautsch. ASVspoof 2019: Automatic Speaker Verification Spoofing and Countermeasures Challenge Evaluation Plan. InASVspoof 2019 Workshop, volume 13, 2019

    2019

  48. [48]

    B. Han, X. Han, H. Zhang, J. Li, and X. Cao. Fighting Fake News: Two Stream Network for Deepfake Detection via Learnable SRM.IEEE Transactions on Biometrics, Behavior, and Identity Science, 3(3):320–331, 2021. 12

    2021

  49. [49]

    H. Tak, M. Kamble, J. Patino, M. Todisco, and N. Evans. RawBoost: A Raw Data Boosting and Augmentation Method Applied to Automatic Speaker Verification Anti-spoofing. InPro- ceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6382–6386. IEEE, 2022

    2022

  50. [50]

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed. Hu- BERT: Self-supervised Speech Representation Learning by Masked Prediction of Hidden Units. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR), pages 1418–1427. IEEE, 2021

    2021

  51. [51]

    Zhang, S

    Q. Zhang, S. Wen, and T. Hu. Audio Deepfake Detection with Self-supervised XLS-R and SLS Classifier. InProceedings of the 32nd ACM International Conference on Multimedia (ACM MM), pages 6765–6773. ACM, 2024

    2024

  52. [52]

    H. Tak, J. Pati˜no, M. Todisco, A. Nautsch, N. Evans, and A. Larcher. End-to-end Anti-Spoofing with RawNet2. InProceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6369–6373. IEEE, 2021

    2021

  53. [53]

    X. Liu, M. Liu, L. Wang, K. A. Lee, H. Zhang, and J. Dang. Leveraging Positional-related Local-global Dependency for Synthetic Speech Detection. InProceedings of the IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023

    2023

  54. [54]

    Y . Chen, J. Yi, J. Xue, C. Wang, X. Zhang, S. Dong, S. Zeng, J. Tao, L. Zhao, C. Fan, and others. RawBMamba: End-to-end Bidirectional State Space Model for Audio Deepfake De- tection. arXiv:2406.06086, 2024

    Pith/arXiv arXiv 2024

  55. [55]

    Rosello, A

    E. Rosello, A. Gomez-Alanis, A. M. Gomez, and A. Peinado. A Conformer-Based Classifier for Variable-Length Utterance Processing in Anti-Spoofing. InProceedings of Interspeech, pages 5281–5285, 2023

    2023

  56. [56]

    N. M. M ¨uller, P. Czempin, F. Dieckmann, A. Froghyar, and K. B¨ottinger. Does Audio Deepfake Detection Generalize? InProceedings of Interspeech, pages 2973–2977. ISCA, 2022

    2022

  57. [57]

    H. Tak, M. Todisco, X. Wang, J.-W. Jung, J. Yamagishi, and N. Evans. Automatic Speaker Verification Spoofing and Deepfake Detection Using Wav2Vec 2.0 and Data Augmentation. arXiv:2202.12233, 2022. A APPENDIX XLSR ARCHITECTUREOVERVIEW XLSR(Cross-Lingual Speech Representations) is a large-scale multilingual model based on the Wav2Vec 2.0architecture, traine...

    Pith/arXiv arXiv 2022

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.