REVIEW 4 major objections 5 minor 27 references
ATMM-SAGA: Alternating Training for Multi-Module with Score-Aware Gated Attention SASV system
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read SAGA gates ECAPA-TDNN embeddings with an AASIST score and reaches 2.18% SASV-EER on ASVspoof2019 LA.
desk verdict A simple, adoptable fusion with a large reported gain, but the headline numbers are picked on the eval set and the significance claim isn't verifiable; still worth a referee. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is the SAGA gate, defined by $e_{SASV}=g(s_{CM},e_{ASV})=s_{CM}e_{ASV}$, a single scalar countermeasure score scaling every dimension of the normalized speaker embedding before a small classifier emits the final SASV score. The second component is ATMM, an alternating training algorithm that at each step randomly freezes the countermeasure branch (with $\lambda=0.1$) or the speaker-verification branch (with $\lambda=0.9$) and updates only the unfrozen module, which is meant to keep both tasks balanced and prevent overfitting to the training attacks. A third detail is the tReLU activation, an affine layer followed by ReLU with a learnable structural matrix initialized as the identity, used on the CM processing path.
What would settle it
Group the ASVspoof2019 LA evaluation trials by unseen attack type (A07-A19) and recompute SASV-EER per attack for the S1 system; if the gate's margin over score fusion disappears on attacks where the AASIST score is miscalibrated, then the reported gain depends on countermeasure reliability rather than on a general fusion advantage. A simpler check is to feed the same architecture a deliberately miscalibrated score and observe whether the SASV-EER degrades.
Extended reading notes
Core claim
The central claim is that a countermeasure score used as a multiplicative gate on the speaker embedding is a better fusion scheme for spoofing-aware speaker verification than score fusion or than applying the gate later in the network. In the S1 strategy the AASIST score $s_{CM}$ is multiplied element-wise with the L2-normalized ECAPA-TDNN embedding: $e_{SASV}=s_{CM}e_{ASV}$. This is intended to suppress spoofed utterances, where $s_{CM}$ is close to zero, and to keep bona fide utterances, where $s_{CM}$ is close to one. The paper reports that this gives SASV-EER of 2.31% on the development set and 2.18% on the evaluation set of ASVspoof2019 LA, with min a-DCF values of 0.0603 and 0.0480. It also reports that the bootstrap confidence intervals for these metrics lie entirely below those of the SASV2022 baselines, the G-SASV system, the standalone ECAPA-TDNN and AASIST systems, and the score-fusion variant S3, which the authors read as statistical evidence for the improvement.
Load-bearing premise
The load-bearing premise is that the countermeasure score is a trustworthy measure of whether each test utterance is genuine or spoofed for every attack type; if a new attack fools the countermeasure, the gate lets the spoof through, and if the countermeasure mislabels genuine speech, the gate suppresses the true speaker.
Editorial extensions
If this is right
- Embedding-level gating with a countermeasure score is a stronger fusion strategy than score fusion for spoofing-aware speaker verification, so future SASV designs can keep pre-trained extractors fixed and improve fusion at the embedding level.
- ATMM-style alternating freezing can substitute for batch normalization and dropout as a way to prevent overfitting to seen spoofing attacks, making the training schedule itself a regularizer.
- The confidence-interval separation reported on the evaluation set implies the improvement over the baselines is not merely run-to-run noise on this benchmark.
- Early gating (S1) being competitive with or better than late gating (S2) suggests that suppressing spoofing information before the final classifier is more effective than after it.
Reading between the lines
- A natural extension is a per-frame or per-band gating vector instead of a single scalar score; the scalar gate already helps, so a richer gate could help on attacks that are detectable only in parts of the signal.
- Because the paper does not report per-attack results on the 11 unseen evaluation attacks, another test is to compute SASV-EER separately for attacks A07-A19; if the advantage is concentrated in attacks AASIST already handles well, the gate's contribution on genuinely novel attacks is still unproven.
- Comparing ATMM directly against modality dropout or stochastic depth would clarify whether the benefit comes from the freezing pattern itself or from the changed loss weighting between the two tasks.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ATMM-SAGA, a spoofing-robust speaker verification system that fuses a pre-trained ECAPA-TDNN speaker embedding extractor with a pre-trained AASIST countermeasure via a score-aware gated attention (SAGA) mechanism. The CM score multiplicatively gates the ASV embeddings (Eq. 2), and the system is trained with an alternating multi-module scheme (ATMM) that freezes one branch while training the other. Experiments on ASVspoof2019 LA report SASV-EER/min a-DCF of 2.31%/0.0603 on the development set and 2.18%/0.0480 on the evaluation set for the best chosen configuration, with additional comparisons to score fusion and prior SASV baselines. The authors conclude that SAGA outperforms score fusion and that ATMM yields better generalization than standard regularization.
Significance. If the reported results are taken at face value, the proposed system is competitive with recent SASV approaches on a standard benchmark, and the ATMM training strategy is an interesting alternative to joint multi-task training with regularization. The paper is clearly written, includes a public code link, and reports bootstrap confidence intervals for the proposed configurations. However, the central quantitative claim is weakened by a post-hoc evaluation-set model selection procedure, and the claimed statistical significance over baselines is not verifiable from the reported tables. The methodological issues bear directly on how the headline numbers should be interpreted, so the contribution cannot be fully assessed without revision.
major comments (4)
- [Section 5.1, Table 2] The best configuration (ATMM without BN/dropout) is selected on the evaluation set, as the text states it 'resulted in the lowest min a-DCF and SASV-EER scores on the evaluation set.' This makes the headline 2.18% EER and 0.0480 a-DCF the best of eight configurations chosen by the held-out test data, rather than an unbiased estimate of a single system's generalization performance. The development-set results tell a different story: the no-ATMM row has a lower dev EER (1.46% vs. 2.31%), so a conventional dev-based model selection would not pick the reported configuration. Please re-run the comparison using development-based selection only, or provide a proper nested protocol that does not use the eval labels for configuration choice, and report all configurations' results without labeling one as the final system based on eval performance.
- [Section 4, Algorithm 1] Training details are insufficient for reproducibility. Algorithm 1 specifies 'for 100 iterations' and a 1% sample of each dataset per branch, but the paper does not report the total number of ATMM rounds, the optimizer, learning rate(s) and schedule, batch size, whether the 100-iteration loop is per epoch or per round, or the initialization and architecture dimensions of the fully connected layers. Since the central claim is empirical and depends on this training procedure, these details should be added to the experimental setup or an appendix.
- [Section 5.2, Table 3] The claim that 'the confidence intervals for the employed evaluation metrics on the evaluation dataset for the proposed solution lie entirely below those of the baseline and individual systems' is not verifiable. Table 3 reports confidence intervals only for S1, S2, and S3; no confidence intervals are given for ECAPA-TDNN, AASIST, Baseline1, Baseline2, or G-SASV. Without those intervals, the statistical-significance statement cannot be checked. Please either provide the bootstrapped CIs for all systems or revise the claim to only compare SAGA against S3, for which intervals are available.
- [Section 3.1.1, Eq. (2)] The gating operation eSASV = sCM * eASV relies on the assumption that sCM is a well-calibrated spoofing probability for unseen attacks. The paper does not provide calibration analysis or a per-attack breakdown on the evaluation set, where 11 of the attacks are unseen. A per-attack table of SASV-EER or false-acceptance rates would show whether the overall 2.18% EER is driven by robust behavior across all unseen attacks or by a few easy attack types. This analysis would also substantiate the intuitive claim that sCM ≈ 0 suppresses spoofed samples and sCM ≈ 1 preserves bona fide samples.
minor comments (5)
- [Throughout] There are multiple typographical errors: 'V oxCeleb1' (Section 3.2.1), 'without BN and withoutdropout' (Section 5.1), and inconsistent spacing around 'dropout' in Table 2's caption. Please proofread.
- [Section 3.2.1] The dataset names 'VoxCeleb1 E and H partitions' are mentioned, but no citation or description of VoxCeleb1 is given beyond the name. Adding a reference and trial construction details would improve reproducibility.
- [Table 3] The G-SASV row reports only the eval SASV-EER (8.62%) with no development value and no a-DCF value. Please clarify whether these numbers were taken directly from [14] and, if so, state the matching protocol (including whether CIs were available in that work).
- [Section 2.3] Equation (1) defines tReLU with the affine layer W_i x + b_i, but the subsequent text says 'Same definition with a diagonal constraint of W_a was implemented in [14–16].' It is unclear whether the proposed system uses the full matrix or a diagonal constraint. Please specify which variant is used in the experiments.
- [Section 5.1] The phrase 'a series of experiments were conducted' should be 'a series of experiments was conducted,' and the sentence starting 'The results in Table 2 indicate that while conventional regularization techniques...' is missing a main clause after 'effective.' Please rephrase.
Circularity Check
No circularity: the reported results are empirical benchmark numbers produced by a trained model, not derivations from their own inputs.
full rationale
The paper makes no formal derivation; it proposes an architecture and reports benchmark numbers. Equation (2) defines eSASV = sCM * eASV as an architectural gating operation, not as a prediction derived from a fitted parameter. The CM score sCM is produced by the trained countermeasure head from the input, so the final SASV score is not equivalent by construction to a fitted constant. The tReLU activation is imported from external prior work, and its use is not load-bearing for the headline claim. The authors' self-citations ([10], [11], [19]) are background or metric citations and are not used to force the central claim; the a-DCF metric is published externally. The comparison against baselines is empirical. A methodological concern, that the best configuration in Table 2 is selected using evaluation-set numbers, affects whether the 2.18% EER is an unbiased generalization estimate, but it is not circularity in the sense of a prediction reducing to its inputs by construction. No quoted equation or citation reduces the central result to the paper's own definitions.
Assumptions & free parameters
free parameters (5)
- ATMM lambda values =
0.1 and 0.9
- Per-iteration dataset sampling ratio =
1%
- Iterations per ATMM round =
100
- Number of ATMM rounds =
not reported
- Fully connected layer dimensions =
not reported
assumptions (5)
- domain assumption Frozen pretrained ECAPA-TDNN and AASIST provide sufficient and complementary representations for SASV.
- domain assumption The CM score sCM is a calibrated and reliable spoofing indicator, including for unseen attacks.
- domain assumption Evaluation-set results were not used to select hyperparameters.
- ad hoc to paper Alternating weight freezing preserves learned knowledge and prevents overfitting.
- domain assumption tReLU is an appropriate activation for the fusion layers.
Cite this review
Pith. "Pith review of ATMM-SAGA: Alternating Training for Multi-Module with Score-Aware Gated Attention SASV system." pith.science (2026). https://pith.science/paper/OT2AIWDF
@misc{pith2026250518273,
author = {Pith},
title = {Pith review of: ATMM-SAGA: Alternating Training for Multi-Module with Score-Aware Gated Attention SASV system},
year = {2026},
howpublished = {\url{https://pith.science/paper/OT2AIWDF}},
note = {Machine review of arXiv:2505.18273}
}
read the original abstract
The objective of automatic speaker verification (ASV) systems is to determine whether a given test speech utterance corresponds to a claimed enrolled speaker. These systems have a wide range of applications, and ensuring their reliability is crucial. In this paper, we propose a spoofing-robust automatic speaker verification (SASV) system employing a score-aware gated attention (SAGA) fusion scheme, integrating scores from a pre-trained countermeasure (CM) with speaker embeddings from a pre-trained ASV. Specifically, we employ the AASIST and ECAPA-TDNN models. SAGA acts as an adaptive gating mechanism, where the CM score determines how strongly ASV embeddings influence the final SASV decision. Experiments on the ASVspoof2019 logical access dataset demonstrate that the proposed SASV system achieves an SASV equal error rate (SASV-EER) and agnostic detection cost function (a-DCF) of 2.31%, 0.0603 for the development set and 2.18%, 0.0480 for the evaluation set.
Figures
Reference graph
Works this paper leans on
-
[1]
Introduction Recent studies have demonstrated that ASV systems are un- dergoing a gradual evolution, acquiring the capacity to reject spoofed inputs in a zero-shot manner. However, rapid advance- ments in speech synthesis techniques, such as text-to-speech (TTS) or voice conversion (VC), highlight the ongoing neces- sity to further enhance spoofing-robust...
-
[2]
Background This section presents a concise review of the pertinent literature on the proposed solution, along with a concise overview of the ASV and CM systems employed for embedding extraction. 2.1. Automatic speaker verification system In this study, the emphasized channel attention, propagation, and aggregation time delay neural network (ECAPA-TDNN) sp...
-
[3]
Proposed system In this section, we will present the operation of the proposed system, which has been designed to address the SASV problem. Given a pair of utterances, an enrollment utterance Uerl of the target speaker and a test utteranceUtst, the system will evaluate whether Utst was spoken by the target speaker (output y = 1) or by a non-target speaker...
-
[4]
Experimental setup In the following, we describe the experimental setup for our pro- posed SASV system, detailing the datasets used for evaluation and the metrics employed to assess performance. 4.1. Dataset The ASVspoof 2019 LA dataset is a widely used bench- mark [18]. This dataset comprises genuine speech utterances and those that have been spoofed usi...
work page 2019
-
[5]
Results In this section, we present a summary of our experimental find- ings. The present study commences with an examination of the influence of distinct training methodologies on model perfor- mance and its generalizability to unseen attacks. In the subse- quent step, an evaluation of the various strategies for integrating the CM score is conducted, wit...
-
[6]
Conclusions and Future Work This paper presents a robust SASV system that integrates CM scores with speaker embeddings using the SAGA mechanism and ATMM algorithm. The proposed approach enables seam- less fusion of ASV and CM pre-trained models while maintain- ing a compact and efficient structure. Our results demonstrate that SAGA is the superior method ...
-
[7]
Acknowledgments This work is supported by the Israel Innovation Authority under project numbers 82457 and 82458
-
[8]
To what ex- tent can ASV systems naturally defend against spoofing attacks?
J. weon Jung, X. Wang, N. Evans, S. Watanabe, H. jin Shim, H. Tak, S. Arora, J. Yamagishi, and J. S. Chung, “To what ex- tent can ASV systems naturally defend against spoofing attacks?” in Interspeech 2024, 2024, pp. 3240–3244
work page 2024
Show all 27 references
-
[9]
SASV 2022: The first spoofing-aware speaker verification challenge,
J. weon Jung, H. Tak, H. jin Shim, H.-S. Heo, B.-J. Lee, S.-W. Chung, H.-J. Yu, N. Evans, and T. Kinnunen, “SASV 2022: The first spoofing-aware speaker verification challenge,” in Interspeech 2022, 2022, pp. 2893–2897
2022
-
[10]
Golden gemini is all you need: Finding the sweet spots for speaker verifica- tion,
T. Liu, K. A. Lee, Q. Wang, and H. Li, “Golden gemini is all you need: Finding the sweet spots for speaker verifica- tion,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 32, pp. 2324–2337, 2024
2024
-
[11]
ECAPA- TDNN: Emphasized channel attention, propagation and aggre- gation in tdnn based speaker verification,
B. Desplanques, J. Thienpondt, and K. Demuynck, “ECAPA- TDNN: Emphasized channel attention, propagation and aggre- gation in tdnn based speaker verification,” in Interspeech 2020, ser. interspeech-2020. ISCA, Oct. 2020. [Online]. Available: http://dx.doi.org/10.21437/Interspee...
2020 doi
-
[12]
AASIST: Audio anti-spoofing using in- tegrated spectro-temporal graph attention networks,
J.-w. Jung, H.-S. Heo, H. Tak, H.-j. Shim, J. S. Chung, B.-J. Lee, H.-J. Yu, and N. Evans, “AASIST: Audio anti-spoofing using in- tegrated spectro-temporal graph attention networks,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing...
2022
-
[13]
End-to-End Spectro-Temporal Graph Attention Net- works for Speaker Verification Anti-Spoofing and Speech Deep- fake Detection,
H. Tak, J. weon Jung, J. Patino, M. Kamble, M. Todisco, and N. Evans, “End-to-End Spectro-Temporal Graph Attention Net- works for Speaker Verification Anti-Spoofing and Speech Deep- fake Detection,” in Proceedings of Interspeech 2021, September 2021
2021
-
[14]
AASIST3: KAN-enhanced AASIST speech deepfake detection using SSL features and additional regularization for the ASVspoof 2024 Challenge,
K. Borodin, V . Kudryavtsev, D. Korzh, A. Efimenko, G. Mkrtchian, M. Gorodnichev, and O. Y . Rogov, “AASIST3: KAN-enhanced AASIST speech deepfake detection using SSL features and additional regularization for the ASVspoof 2024 Challenge,” in The Automatic Speaker Verification ...
2024
-
[15]
On the potential of jointly-optimised solutions to spoofing attack detection and auto- matic speaker verification,
W. Ge, H. Tak, M. Todisco, and N. Evans, “On the potential of jointly-optimised solutions to spoofing attack detection and auto- matic speaker verification,” in IberSPEECH 2022, 2022, pp. 51– 55
2022
-
[16]
Integrated Spoofing Countermeasures and Automatic Speaker Verification: An Evaluation on ASVspoof 2015,
M. Sahidullah, H. Delgado, M. Todisco, H. Yu, T. Kinnunen, N. Evans, and Z.-H. Tan, “Integrated Spoofing Countermeasures and Automatic Speaker Verification: An Evaluation on ASVspoof 2015,” in Proceedings of the 17th Annual Conference of the International Speech Communication ...
2015
-
[17]
Spoofing-Robust Speaker Verification Based on Time-Domain Embedding,
A. Weizman, Y . Ben-Shimol, and I. Lapidot, “Spoofing-Robust Speaker Verification Based on Time-Domain Embedding,” in Cyber Security, Cryptology, and Machine Learning, S. Dolev, M. Elhadad, M. Kutyłowski, and G. Persiano, Eds. Cham: Springer Nature Switzerland, 2025, pp. 64–78
2025
-
[18]
Tandem spoofing-robust automatic speaker verification based on time-domain embeddings,
——, “Tandem spoofing-robust automatic speaker verification based on time-domain embeddings,” 2024. [Online]. Available: https://arxiv.org/abs/2412.17133
2024 arXiv
-
[19]
A probabilistic fusion frame- work for spoofing aware speaker verification,
Y . Zhang, G. Zhu, and Z. Duan, “A probabilistic fusion frame- work for spoofing aware speaker verification,” inThe Speaker and Language Recognition Workshop (Odyssey 2022), 2022, pp. 77– 84
2022
-
[20]
HYU Submission for the SASV Challenge 2022: Reforming Speaker Embeddings with Spoofing-Aware Conditioning,
J.-H. Choi, J.-Y . Yang, Y .-R. Jeoung, and J.-H. Chang, “HYU Submission for the SASV Challenge 2022: Reforming Speaker Embeddings with Spoofing-Aware Conditioning,” in Interspeech 2022, 2022, pp. 2873–2877
2022
-
[21]
Generalizing speaker verification for spoof awareness in the embedding space,
X. Liu, M. Sahidullah, K. A. Lee, and T. Kinnunen, “Generalizing speaker verification for spoof awareness in the embedding space,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 32, p. 1261–1273, 2024. [Online]. Available: http://dx.doi.org/10.1109/TASLP...
2024
-
[22]
DNN speaker adaptation using parameterised sigmoid and ReLU hidden activation functions,
C. Zhang and P. C. Woodland, “DNN speaker adaptation using parameterised sigmoid and ReLU hidden activation functions,” in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2016, pp. 5300–5304
2016
-
[23]
Adaptation algorithms for neural network-based speech recognition: An overview,
P. Bell, J. Fainberg, O. Klejch, J. Li, S. Renals, and P. Swietojanski, “Adaptation algorithms for neural network-based speech recognition: An overview,” IEEE Open Journal of Signal Processing, vol. 2, p. 33–66, 2021. [Online]. Available: http://dx.doi.org/10.1109/OJSP.2020.3045349
2021
-
[24]
Joint Decision of Anti-Spoofing and Automatic Speaker Verification by Multi-Task Learning With Contrastive Loss,
J. Li, Z. Wu, J. Dang, and H. Li, “Joint Decision of Anti-Spoofing and Automatic Speaker Verification by Multi-Task Learning With Contrastive Loss,” IEEE Access, vol. 8, pp. 58 534–58 542, 2020
2020
-
[25]
ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech,
X. Wang, J. Yamagishi, M. Todisco, H. Delgado, A. Nautsch, N. Evans, M. Sahidullah, V . Vestman, T. Kinnunen, K. A. Lee, L. Juvela, P. Alku, Y .-H. Peng, H.-T. Hwang, Y . Tsao, H.-M. Wang, S. L. Maguer, M. Becker, F. Henderson, R. Clark, Y . Zhang, Q. Wang, Y . Jia, K. Onuma, ...
2019
-
[26]
a-DCF: an architecture agnostic metric with applica- tion to spoofing-robust speaker verification,
H.-J. Shim, J.-W. Jung, T. Kinnunen, N. Evans, J.-F. Bonastre, and I. Lapidot, “a-DCF: an architecture agnostic metric with applica- tion to spoofing-robust speaker verification,” inOdyssey 2024, 06 2024
2024
-
[27]
Confidence intervals for evaluation in machine learning
L. Ferrer and P. Riera, “Confidence intervals for evaluation in machine learning.” [Online]. Available: https://github.com/ luferrer/ConfidenceIntervals
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.