REVIEW 4 major objections 5 minor 30 references
Synthetic Speech Source Tracing using Metric Learning
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A 1.4M-parameter ResNet with the GE2E speaker-verification loss can trace a synthetic speech clip to its generating TTS system about as accurately as a 315M-parameter SSL model.
desk verdict Useful benchmark, but the 'ResNet beats SSL' claim is confounded by unequal training budgets and no error bars. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the combination of a Thin-ResNet-34 encoder (about 1.4M parameters) with the Generalized End-to-End (GE2E) loss, a metric-learning objective borrowed from speaker verification. GE2E forms per-class centroids within a batch, computes scaled cosine similarities between each query embedding and all centroids, and applies a softmax over classes; this forces the model to place utterances by the same TTS system close together and different systems apart. The paper also uses self-attentive pooling to turn frame-level features into a single utterance embedding, and compares against an AASIST graph-attention backbone fed with fine-tuned Wav2Vec2 embeddings.
What would settle it
Train the AASIST + Wav2Vec2 model for 300 epochs with a batch size of 128 (or with the same total compute as the ResNet) on the same source-tracing split, and check whether its test EER falls below the ResNet's best 4.63%; if it does, the paper's 'matching and exceeding' claim fails under equal budgets.
Extended reading notes
Core claim
The authors' central claim is that a simple ResNet model with a GE2E loss and 50-dimensional embeddings is a viable solution to discern between the source model of a spoofed sample. They show that with cosine-similarity scoring, the best ResNet configuration (AAMSoftmax, 50-dim, random sampler) reaches 4.63% EER on the held-out MLAADv5 test set, while the best SSL configuration (AASIST with fine-tuned Wav2Vec2, AAMSoftmax, 10-dim) reaches 4.77% EER. They further argue that metric-learning losses like GE2E are particularly effective on the ResNet, with all embedding sizes below 5.5% test EER, and that linear probing of the learned embeddings shows good class separability even for unseen TTS systems.
Load-bearing premise
The ResNet and SSL models are compared as if they had equal training opportunity, but the ResNet was trained three times longer (300 vs. 100 epochs) and with four times the batch size; if the SSL model received a comparable budget, the observed performance parity could disappear.
Editorial extensions
If this is right
- A 1.4M-parameter model trained on a single GPU could serve as a practical source-tracing tool, lowering the compute and memory barrier for forensic deployment.
- The GE2E loss transfers from speaker verification to synthetic speech attribution, so a closed-set classification head is unnecessary and embeddings generalize to unseen TTS systems.
- Embedding sizes of 10–50 dimensions suffice; larger 200–512 dimensions bring no gain, so storage and matching costs stay low.
- SSL backbones for source tracing are currently under-trained relative to the ResNet; equalized training budgets could change the measured ranking.
Reading between the lines
- If SSL models were trained for as many epochs with as large batches as the ResNet, their test EER might drop below the ResNet's 4.63%; a direct comparison with equal compute would separate architecture potential from training schedule.
- The confusion between mms-tts language variants and between vits and griffin-lim pairs suggests source identity is partly a language-plus-architecture fingerprint; a model that disentangles these factors could improve attribution.
- The better generalization of Mel filterbanks over Wav2Vec2 embeddings to unseen classes hints that low-level acoustic traces carry the source identity, motivating a study of which frequency bands encode the fingerprint.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses synthetic speech source tracing, i.e., identifying which TTS system generated a given spoofed audio sample, using speaker-recognition-style pipelines. The authors evaluate Thin-ResNet-34 and an SSL-based model (Wav2Vec2 + AASIST) on the MLAADv5 source-tracing benchmark, comparing classification losses (Softmax, AMSoftmax, AAMSoftmax) and metric-learning losses (GE2E, Angular Prototypical) across embedding dimensions of 10, 50, 200, and 512. The central claim, stated in the Abstract and Conclusions, is that a lightweight ResNet with metric learning is competitive with, and in some configurations exceeds, a much larger self-supervised model, as measured by EER on the test set. The paper also includes linear probing and UMAP visualizations to support the utility of the learned embeddings.
Significance. If the claim were rigorously established, the paper would make a useful contribution to audio forensics: it would show that a 1.4M-parameter ResNet trained with straightforward losses can compete with a 315M-parameter SSL/AASIST pipeline for source tracing, enabling lightweight practical deployments and providing a strong baseline for future work. The paper also usefully catalogues a broad set of loss/sampler/embedding-size combinations on a public benchmark. However, the significance is currently provisional because the central comparison rests on unequal training budgets, post-hoc configuration selection, and single runs without error bars. The paper does not provide code, pretrained models, or machine-checked proofs, but it does use an external benchmark (MLAADv5) with a development/test split, which is a strength in terms of evaluation design.
major comments (4)
- [§4.4, Abstract, Table 1] The 'matching and even exceeding' claim in the Abstract is not supported by the experimental protocol because the ResNet and SSL models are trained with unequal budgets: ResNet runs use 300 epochs and batch size 128, while SSL runs use 100 epochs and batch size 32 (Section 4.4). This is a confound: the SSL model may simply be undertrained, so the observed parity (e.g., AASIST AAMSoftmax R 10-dim 4.77% vs. ResNet AAMSoftmax R 50-dim 4.63%) could vanish under a comparable budget. The authors should either give the SSL model a comparable number of optimization steps (not just wall-clock hours, which are dominated by the larger model) or explicitly frame the comparison as 'under a fixed wall-clock budget' and temper the Abstract claim accordingly.
- [§4.5, §5, Table 1, Conclusions] The model-selection protocol is under-specified and appears to be post-hoc. Section 4.5 states that development EER is computed during training, but no early-stopping or model-selection rule is described. The best test result (AAMSoftmax R, 50-dim, 4.63%) is not the best development result (GE2E B 50-dim, 4.57%), and the configuration highlighted in the Conclusions (GE2E, 50-dim) gives 5.03% test EER, not the 'exceeding' result quoted in the Abstract. This internal inconsistency means the reader cannot tell which configuration is the intended final model. The authors should specify a single selection rule (e.g., pick the model with lowest development EER per architecture/loss family) and report test EER for that rule, rather than cherry-picking the best test result after evaluation.
- [Table 1, §5] All reported EERs come from single runs with no error bars or repeated-seed experiments. Several claims rest on differences of 0.1-0.3 percentage points (e.g., ResNet AAMSoftmax R 50-dim 4.63% vs. AASIST AAMSoftmax R 10-dim 4.77%; ResNet GE2E 200-dim 4.94% vs. 50-dim 5.03%). Without variance estimates, the reader cannot judge whether these differences are meaningful. The authors should provide at least two or three seeds per configuration and report mean±std, or a bootstrap confidence interval, particularly for the configurations that support the 'exceeding' claim.
- [§2.2, §4.4, Title] The title and framing emphasize metric learning, but the SSL/AASIST model is only trained with AMSoftmax and AAMSoftmax losses (classification losses), not with GE2E or Angular Prototypical. This means the comparison between 'metric learning' (ResNet) and 'SSL-based systems' (AASIST) is not symmetric: the SSL backbone is never tested with the metric-learning losses that are the paper's focus. The conclusion that 'SSL representations need optimization for this task' would be stronger if the same metric losses were applied to the SSL backbone. At minimum, the paper should acknowledge this limitation and avoid implying that metric learning was evaluated across both backbones.
minor comments (5)
- [Abstract] The phrase 'matching and even exceeding SSL-based systems' overstates the evidence in Table 1, where only a single ResNet configuration exceeds the best AASIST configuration and by only 0.14 percentage points without error bars; suggest a more cautious wording such as 'competitive with'.
- [§4.4] The sentence 'The ResNet models’ trained for about 2 hours' contains a typo ('models’' should be 'models were'); also, 'V oice Activity Detection' has an extra space in Section 4.1.
- [§5] The statement 'which is surprising because, on the development set, this configuration was the only one below 6% EER' is confusing: Table 1 shows many development entries below 6%, including GE2E, AngularProto, and AASIST entries; please clarify which configuration and which subset of results is being referred to.
- [§5, Fig. 1] The caption mentions 'marked with a red circle', but no red circles are visible in the figure; either add them or adjust the caption.
- [§4.5] The linear probing section describes undersampling and an 80/20 split of the development embeddings, but it omits whether the linear probe was evaluated on the held-out 20% or the full development set; this should be clarified for reproducibility.
Circularity Check
No circularity: the paper's claims are empirical evaluations against an external test set using prior-work losses and standard architectures.
full rationale
The paper's central claims are empirical results on an external benchmark (the MLAADv5 source-tracing split): ResNet with speaker-recognition-inspired losses achieves test EER comparable to an AASIST+Wav2Vec2 SSL backbone. No step in the paper derives a conclusion from an input that already contains it. The loss functions (Eqs. 1-9) are explicitly attributed to prior work ([8]-[13], [15]) and are not redefined in terms of the outcome being predicted. The models are standard architectures (Thin-ResNet-34 and AASIST with Wav2Vec2) trained on the training split; development EER is used for monitoring and test EER is reported on a mutually exclusive test subset (Section 3), so the reported numbers are not fitted parameters renamed as predictions. No self-citation is load-bearing: the cited repository and related speaker-recognition works are by other research groups, and no "uniqueness theorem" or prior result by the present authors is invoked to force a choice. Potential weaknesses such as unequal training budgets between ResNet and SSL models or post-hoc selection of the best test configuration are experimental-design and fairness concerns, not circularity: they do not make any equation or claim equivalent to its own input by construction. The derivation chain is therefore self-contained against an external benchmark, and no circular step can be exhibited.
Assumptions & free parameters
free parameters (2)
- AMSoftmax/AAMSoftmax margin m =
0.3
- AMSoftmax/AAMSoftmax scale s =
30
assumptions (4)
- domain assumption MLAADv5 source tracing benchmark is a valid proxy for real-world source tracing.
- domain assumption Pairwise cosine similarity EER is an appropriate evaluation metric for open-set source tracing.
- standard math The losses and architectures from prior work (GE2E, Angular Prototypical, AAMSoftmax) are correctly implemented.
- domain assumption 2-second audio segments preserve source identity information.
Cite this review
Pith. "Pith review of Synthetic Speech Source Tracing using Metric Learning." pith.science (2026). https://pith.science/paper/WXZJEAYJ
@misc{pith2026250602590,
author = {Pith},
title = {Pith review of: Synthetic Speech Source Tracing using Metric Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/WXZJEAYJ}},
note = {Machine review of arXiv:2506.02590}
}
read the original abstract
This paper addresses source tracing in synthetic speech-identifying generative systems behind manipulated audio via speaker recognition-inspired pipelines. While prior work focuses on spoofing detection, source tracing lacks robust solutions. We evaluate two approaches: classification-based and metric-learning. We tested our methods on the MLAADv5 benchmark using ResNet and self-supervised learning (SSL) backbones. The results show that ResNet achieves competitive performance with the metric learning approach, matching and even exceeding SSL-based systems. Our work demonstrates ResNet's viability for source tracing while underscoring the need to optimize SSL representations for this task. Our work bridges speaker recognition methodologies with audio forensic challenges, offering new directions for combating synthetic media manipulation.
Figures
Reference graph
Works this paper leans on
-
[1]
Introduction Recent advances in audio manipulation, particularly text-to- speech (TTS) models [1, 2, 3] have enabled synthetic media to be indistinguishable from human speech. However, these innovations have also been exploited for malicious purposes [4], driving the need for robust anti-spoofing systems. Al- though detecting spoofed audio remains paramou...
-
[2]
Losses and metric learning methods We now present the methods tested during our research, catego- rized into two groups: classification loss functions and metric learning methods. 2.1. Classification Loss Functions The training set of the MLAADv5 [19] dataset, and more specifically a version for source tracing 1, contains T = 24 distinct TTS systems. Duri...
arXiv 2025
-
[3]
Dataset The dataset used in this source tracing task is a version of the MLAADv5 dataset designed specifically for source tracing. The dataset consists of three distinct and mutually exclusive subsets, a training set, a development set, and a test set. The training subset features 11.100 audio samples across 24 distinct classes, each class representing th...
-
[4]
Experimental Setup For the purposes of this research we utilized the repository2 the authors of [15] released, which we altered in order to work for this project, while enhancing its model pool with AASIST with fine-tuned Wav2Vec2 embeddings. 4.1. Inputs During training, as input, we extracted randomly a 2s segment from each audio file. The input embeddin...
-
[5]
Experimental Results The results for both the ResNet and the AASIST models can be found in Table 1. The ResNet model exhibited mixed perfor- mance when trained with different classification loss functions. Notably, the softmax loss with a random sampler struggled to differentiate between audio samples across most output embed- ding sizes, with the excepti...
-
[6]
Conclusions We present an approach to the synthetic speech source tracing task that is largely inspired by the speaker recognition pipeline. Our experiments show that a simple ResNet model with a GE2E loss and 50-dim embeddings is a viable solution to discern be- tween the source model of a spoofed sample. We also provide evidence that the size of the emb...
-
[7]
Acknowledgements This work has been partially supported by project MIS 5154714 of the National Recovery and Resilience Plan Greece 2.0 funded by the European Union under the NextGenerationEU Program
-
[8]
Xtts: a massively multilingual zero-shot text-to-speech model,
E. Casanova, K. Davis, E. G ¨olge, G. G ¨oknar, I. Gulea, L. Hart, A. Aljafari, J. Meyer, R. Morais, S. Olayemi, and J. Weber, “Xtts: a massively multilingual zero-shot text-to-speech model,” in In- terspeech 2024, 2024, pp. 4978–4982
work page 2024
Show all 30 references
-
[9]
Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,
J. Kim, J. Kong, and J. Son, “Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,” in Pro- ceedings of the 38th International Conference on Machine Learn- ing, ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Eds., ...
2021
-
[10]
Glow-tts: A generative flow for text-to-speech via monotonic alignment search,
J. Kim, S. Kim, J. Kong, and S. Yoon, “Glow-tts: A generative flow for text-to-speech via monotonic alignment search,” in Ad- vances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., ...
2020
-
[11]
Not my voice! a taxonomy of ethical and safety harms of speech generators,
W. Hutiri, O. Papakyriakopoulos, and A. Xiang, “Not my voice! a taxonomy of ethical and safety harms of speech generators,” in The 2024 ACM Conference on Fairness, Accountability, and Transparency, 2024, pp. 359–376
2024
-
[12]
Source trac- ing of audio deepfake systems,
N. Klein, T. Chen, H. Tak, R. Casal, and E. Khoury, “Source trac- ing of audio deepfake systems,” in Interspeech 2024, 2024, pp. 1100–1104
2024
-
[13]
An initial investigation for detecting vocoder fingerprints of fake audio,
X. Yan, J. Yi, J. Tao, C. Wang, H. Ma, T. Wang, S. Wang, and R. Fu, “An initial investigation for detecting vocoder fingerprints of fake audio,” in Proceedings of the 1st International Work- shop on Deepfake Detection for Audio Multimedia , ser. DDAM ’22. New York, NY , USA: A...
2022
-
[14]
Synthetic speech detection through short-term and long-term prediction traces,
C. Borrelli, P. Bestagini, F. Antonacci, A. Sarti, and S. Tubaro, “Synthetic speech detection through short-term and long-term prediction traces,” EURASIP J. Inf. Secur., vol. 2021, no. 1, pp. 1–14, 2021
2021
-
[15]
Sphereface: Deep hypersphere embedding for face recognition,
W. Liu, Y . Wen, Z. Yu, M. Li, B. Raj, and L. Song, “Sphereface: Deep hypersphere embedding for face recognition,” in2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 6738–6746
2017
-
[16]
Cosface: Large margin cosine loss for deep face recog- nition,
H. Wang, Y . Wang, Z. Zhou, X. Ji, D. Gong, J. Zhou, Z. Li, and W. Liu, “Cosface: Large margin cosine loss for deep face recog- nition,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 5265–5274
2018
-
[17]
Arcface: Additive angular margin loss for deep face recognition,
J. Deng, J. Guo, N. Xue, and S. Zafeiriou, “Arcface: Additive angular margin loss for deep face recognition,” in2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 4685–4694
2019
-
[18]
Generalized end-to-end loss for speaker verification,
L. Wan, Q. Wang, A. Papir, and I. L. Moreno, “Generalized end-to-end loss for speaker verification,” in 2018 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE Press, 2018, pp. 4879–4883
2018
-
[19]
Facenet: A unified embedding for face recognition and clustering,
F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 815–823
2015
-
[20]
Prototypical networks for few-shot learning,
J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot learning,” in Proceedings of the 31st International Con- ference on Neural Information Processing Systems, ser. NIPS’17. Curran Associates Inc., 2017, pp. 4080–4090
2017
-
[21]
From speaker verification to deepfake algorithm recognition: Our learned lessons from add2023 track 3
X. Qin, X. Wang, Y . Chen, Q. Meng, and M. Li, “From speaker verification to deepfake algorithm recognition: Our learned lessons from add2023 track 3.” inDADA@ IJCAI, 2023, pp. 107– 112
2023
-
[22]
In defence of metric learning for speaker recognition,
J. S. Chung, J. Huh, S. Mun, M. Lee, H.-S. Heo, S. Choe, C. Ham, S. Jung, B.-J. Lee, and I. Han, “In defence of metric learning for speaker recognition,” in Interspeech 2020, 2020, pp. 2977–2981
2020
-
[23]
Delving into vox- celeb: environment invariant speaker recognition,
J. S. Chung, J. Huh, and S. Mun, “Delving into vox- celeb: environment invariant speaker recognition,” arXiv preprint arXiv:1910.11238, 2019
1910 arXiv
-
[24]
Aasist: Audio anti-spoofing using in- tegrated spectro-temporal graph attention networks,
J.-w. Jung, H.-S. Heo, H. Tak, H.-j. Shim, J. S. Chung, B.-J. Lee, H.-J. Yu, and N. Evans, “Aasist: Audio anti-spoofing using in- tegrated spectro-temporal graph attention networks,” in ICASSP 2022-2022 IEEE international conference on acoustics, speech and signal processing (...
2022
-
[25]
wav2vec 2.0: A framework for self-supervised learning of speech repre- sentations,
A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech repre- sentations,” Advances in neural information processing systems , vol. 33, pp. 12 449–12 460, 2020
2020
-
[26]
Mlaad: The multi- language audio anti-spoofing dataset,
N. M. M ¨uller, P. Kawa, W. H. Choong, E. Casanova, E. G ¨olge, T. M¨uller, P. Syga, P. Sperl, and K. B¨ottinger, “Mlaad: The multi- language audio anti-spoofing dataset,” in2024 International Joint Conference on Neural Networks (IJCNN), 2024, pp. 1–7
2024
-
[27]
Instance nor- malization: The missing ingredient for fast stylization. corr abs/1607.08022 (2016),
D. Ulyanov, A. Vedaldi, and V . S. Lempitsky, “Instance nor- malization: The missing ingredient for fast stylization. corr abs/1607.08022 (2016),” arXiv preprint arXiv:1607.08022, 2016
2016 arXiv
-
[28]
Exploring the encoding layer and loss function in end-to-end speaker and language recognition system,
W. Cai, J. Chen, and M. Li, “Exploring the encoding layer and loss function in end-to-end speaker and language recognition system,” arXiv preprint arXiv:1804.05160, 2018
2018 arXiv
-
[29]
The tunnel effect: Building data representations in deep neural networks,
W. Masarczyk, M. Ostaszewski, E. Imani, R. Pascanu, P. Mił o ´s, and T. Trzcinski, “The tunnel effect: Building data representations in deep neural networks,” inAdvances in Neural Information Pro- cessing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Le...
2023
-
[30]
Umap: Uni- form manifold approximation and projection,
L. McInnes, J. Healy, N. Saul, and L. Grossberger, “Umap: Uni- form manifold approximation and projection,” The Journal of Open Source Software, vol. 3, no. 29, p. 861, 2018
2018
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.