REVIEW 4 major objections 4 minor 30 references
A Study on Angular Based Embedding Learning for Text-independent Speaker Verification
T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Angular margin losses plus an inter-class regularizer improve text-independent speaker verification on VoxCeleb, cutting equal error rate by 16.5%.
desk verdict Useful comparative study of angular margin losses on VoxCeleb, but the paper's claimed regularizer contribution rests on thin evidence. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the hyperspherical energy regularizer in Eq. (7), $L_{inter} = \frac{1}{C}\| [W_n^T W_n]_+ - I \|_F^2$, applied to the $\ell^2$-normalized columns $W_n$ of the classifier weight matrix $W$. Clamping negative elements to zero and measuring the squared Frobenius norm penalizes any pair of class centers that have positive cosine similarity, leaving orthogonal or antipodal pairs unpenalized. This regularizer is added to the angular-margin loss with weight $\lambda_{inter} = 0.01$, and it is what pushes the class centers into a more uniform arrangement on the hypersphere. The paper also relies on annealing formulas to make the angular losses converge.
What would settle it
Train the same ResNet18 with AM-softmax margin 0.2 on VoxCeleb with $\lambda_{inter} = 0$ and $\lambda_{inter} = 0.01$ across several seeds; if the regularized version does not consistently improve held-out trial EER, or if a model with lower hyperspherical energy on the training classes does not generalize to lower EER on test trials, the transfer claim is refuted.
Extended reading notes
Core claim
The paper claims that angular-margin softmax losses, specifically A-softmax, AM-softmax, and AAM-softmax, are effective for open-set speaker verification, and that their main weakness, poor inter-class separability of the classifier's weight columns, can be corrected by an exclusive inter-class regularizer. The regularizer computes the hyperspherical energy of the $\ell^2$-normalized weight matrix W, penalizing positive pairwise cosine similarities among the class-center vectors and pushing the centers to spread uniformly around the unit hypersphere. With this regularizer added, every angular system improves; the best system, AM-softmax with margin 0.2, reaches EER 4.45% and minDCF 0.400, compared with 5.33% and 0.489 for the ResNet18 softmax baseline. The angular systems also require annealing during training, starting from modified softmax and gradually increasing the angular-margin term, for the networks to converge.
Load-bearing premise
The load-bearing premise is that pushing the classifier's class-center vectors to spread uniformly around the hypersphere transfers into better separability of unseen speaker pairs in the embedding space; if that transfer fails, the regularizer's gains disappear and the headline improvement would come only from the already known angular-margin losses.
Editorial extensions
If this is right
- The best system, AM-softmax with margin 0.2 plus the inter-class regularizer, reduces EER from 5.33% to 4.45% relative to the ResNet18 softmax baseline on VoxCeleb.
- All three angular-margin families, A-softmax, AM-softmax, and AAM-softmax, beat softmax, and adding the regularizer improves each of them, suggesting the effect is not tied to one margin formulation.
- Inter-class regularization lowers the hyperspherical energy of the weight columns and increases between-class angular variance on the test set, implying better open-set separability.
- The gains are strongest at low false-alarm operating points, which matters for security-oriented verification tasks.
Reading between the lines
- If the transfer to other open-set recognition tasks holds, the same regularizer should improve face verification, speaker diarization, and few-shot classification, but the paper only tests VoxCeleb speaker verification.
- The regularizer's effect likely depends on the embedding dimension: in low dimensions, forcing uniform class centers could limit capacity, so varying dimension and $\lambda_{inter}$ together would be a natural extension.
- Because the paper uses cosine scoring without a probabilistic back-end, testing the regularized embeddings with PLDA scoring could show whether the gains are additive or partly redundant.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies angular-margin softmax losses (A-softmax, AAM-softmax, AM-softmax) for text-independent speaker verification on VoxCeleb data. It proposes an inter-class regularization term, L_inter in Eq. (7), that penalizes positive pairwise cosines among l2-normalized classifier weight columns, and reports that adding this regularizer to angular losses improves EER and minDCF. The best system, AM-softmax with margin 0.2 plus the regularizer, reaches EER 4.45% versus 5.33% for a ResNet18 softmax baseline, corresponding to a 16.5% relative EER improvement and 18.2% minDCF improvement. The paper also reports qualitative t-SNE plots, a between-class angular variance metric, score distributions, and a DET plot.
Significance. If the claims are fully supported, the paper would provide a useful controlled comparison of angular-margin losses in speaker verification and a simple regularizer that improves inter-class separability. The comparison of three angular losses under a common protocol is a useful contribution, and the headline improvement over the softmax baseline is an independent empirical result. However, the central novel component, the inter-class regularizer, is not convincingly validated: the reported gains from adding it are small, no uncertainty or significance information is given, and the main supporting metric (SEP_W) is essentially the same quantity that the regularizer directly minimizes. The paper is therefore technically sound in its baseline comparisons but needs stronger evidence for the proposed contribution.
major comments (4)
- [II-C, Eq. (6)-(8), Table II] SEP_W in Eq. (6) and L_inter in Eq. (7) are mathematically equivalent: after l2-normalizing the columns of W, W_n^T W_n has diagonal ones and off-diagonal cosines, so clamping and subtracting I yields exactly the sum of squared positive off-diagonal cosines in Eq. (6), up to the 1/C scaling. Therefore Table II's observation that A-softmax plus inter-class regularization lowers SEP_W is partly true by construction: the network is directly trained to minimize that objective on the training set. To support the claim that the regularizer improves open-set inter-class separability, the paper needs an independent measure, for example evaluation on held-out speakers or on embeddings rather than on the optimized classifier weights.
- [Table I, Section III-A] The incremental benefit of the proposed regularizer is very small and is reported without any statistical support. Adding L_inter changes EER from 4.56 to 4.45 for AM-softmax m=0.2, from 4.55 to 4.49 for AAM-softmax m=0.3, and from 4.58 to 4.46 for A-softmax m=2, while minDCF changes by 0.002 in each case. These are single-run numbers with no error bars, no multiple seeds, and no paired significance test. Given that the VoxCeleb1 trial list has 37,720 trials, the observed differences may lie within seed-level or optimization variability. The paper should report multiple runs with mean and standard deviation, or at minimum a paired test, before claiming that the regularizer consistently improves the angular-margin systems.
- [III-A and III-B, Table I] The evaluation protocol appears to select hyperparameters on the same test set used for the headline numbers. The verification test list is described in Section III-A as the VoxCeleb I verification trial list, and Table I reports results for several margins (m1, m2, m3) and λ_inter = 0.01 without any separate validation set being mentioned. If the margins and λ_inter were chosen by observing EER/minDCF on this trial list, the reported numbers are optimistically biased. The authors should either use a held-out validation set for hyperparameter selection or explicitly state that the reported margins were fixed a priori.
- [III-C, Eq. (9), Fig. 2] The evidence intended to show that the regularizer improves inter-class separability in the embedding space is presented only for A-softmax, not for the best system (AM-softmax with the regularizer). The between-class angular variance Sb and the non-target score distribution plots are reported for A-softmax with and without the regularizer, but the headline result is from AM-softmax plus the regularizer. It would be more compelling to show the same separability analysis for the best system, or to explain why the A-softmax case is representative. Without that, the link between the proposed mechanism and the reported EER gain is weaker than the abstract suggests.
minor comments (4)
- [II-C] The phrase 'ideally this scalar should be minimize to zero' is grammatically unclear; also, since W_n columns are l2-normalized, the diagonal of W_n^T W_n is 1, so the expression in Eq. (7) reduces to a sum over off-diagonal positive cosines; stating this explicitly would improve readability.
- [Table II] There is a typo in the text: 'Talbe II' should be 'Table II'.
- [III-A, Table I] The baseline comparison mixes architectures: Resnet34 Softmax [29] is listed alongside the authors' Resnet18 systems, and it is unclear whether the Resnet18 softmax baseline was trained by the authors or taken from another source. Please label the origin of each row clearly.
- [II-C] The paper cites RegularFace [24] and other inter-class regularization works, but does not explicitly state what is new about the proposed regularization relative to RegularFace's exclusive regularization. A brief comparative paragraph would help position the contribution.
Circularity Check
The SEP_W evidence in Table II is self-definitional: Eq. (6) and the L_inter objective in Eq. (7) are the same quantity up to scaling, so observing lower SEP_W after training with the regularizer restates the training loss. The headline EER/minDCF gains remain independent test-trial measurements.
-
self definitional
[Section II-C, Eqs. (6)-(8); Section III-C, Table II and following paragraph]
"We consider the following separability measurement: SEP_W = 1/C Σ_j Σ_{i,i≠j} max[0, cos(φ_i,j)]^2, (6), where φ_i,j is the angle between w_i and w_j, ideally this scalar should be minimize to zero. This is equivalent to the following criterion: L_inter = 1/C ||[W_n^T W_n]_+ − I||_F^2, (7). ... In Table II, we first compare the hyperspherical energy in equation 6, which reflect the separability between the cluster centers. ... A-softmax with inter-class regularization ... lead to reduced hyperspherical energy. This shows such regularization encourages inter-class separability."
Eq. (6) and Eq. (7) are the same objective up to the 1/C scaling and matrix notation: L_inter is the squared Frobenius norm of the positive off-diagonal entries of W_n^T W_n, which is exactly the sum in SEP_W. The '+ inter' models are trained with Eq. (8), which explicitly includes L_inter, so their weight columns are directly optimized to reduce this value. Reporting decreased SEP_W for these models in Table II is therefore a restatement of the training objective rather than independent evidence of inter-class separability. The paper's other evidence (Sb, non-target score distributions, and Table I EER/minDCF) is measured independently on evaluation trials and does not reduce to the loss definition.
full rationale
The central derivation of the angular margin losses is not circular: A-softmax, AM-softmax, and AAM-softmax are imported from the face-recognition literature and applied to VoxCeleb without fitting any parameter to the test labels. The headline 16.5% EER and 18.2% minDCF improvements are empirical comparisons on the VoxCeleb verification trial list, and the best-system configuration is not derived from those test numbers by construction. The only genuinely circular step is the Table II SEP_W comparison, because SEP_W is exactly the quantity minimized by the proposed regularizer; that observation is true by definition. Since this tautology is supporting evidence for the regularizer rather than the source of the reported EER gains, the overall circularity is moderate, not severe; the statistical fragility of the ~0.1 EER differences is a correctness/robustness concern, not a circularity concern.
Assumptions & free parameters
free parameters (4)
- AM-softmax margin m3 =
0.2
- Inter-class regularization weight lambda_inter =
0.01
- AAM-softmax margin m2 =
0.3
- A-softmax margin m1 =
2
assumptions (4)
- domain assumption A ResNet18 trained on a random 100-excerpt-per-speaker subset of VoxCeleb2 dev, without VAD or augmentation, is a valid proxy for comparing embedding losses on VoxCeleb.
- domain assumption Cosine similarity scoring on 512-d embeddings is a sufficient back-end; no length normalization or PLDA is needed.
- domain assumption Minimizing hyperspherical energy of the classifier weight matrix W (Eq. 7) improves inter-class separability of the test embedding distribution.
- ad hoc to paper The annealing recipes in Eqs. (4)-(5) are necessary and sufficient for convergence of angular losses on ASV.
Cite this review
Pith. "Pith review of A Study on Angular Based Embedding Learning for Text-independent Speaker Verification." pith.science (2026). https://pith.science/paper/VF5LS3UY
@misc{pith2026190803990,
author = {Pith},
title = {Pith review of: A Study on Angular Based Embedding Learning for Text-independent Speaker Verification},
year = {2026},
howpublished = {\url{https://pith.science/paper/VF5LS3UY}},
note = {Machine review of arXiv:1908.03990}
}
read the original abstract
Learning a good speaker embedding is important for many automatic speaker recognition tasks, including verification, identification and diarization. The embeddings learned by softmax are not discriminative enough for open-set verification tasks. Angular based embedding learning target can achieve such discriminativeness by optimizing angular distance and adding margin penalty. We apply several different popular angular margin embedding learning strategies in this work and explicitly compare their performance on Voxceleb speaker recognition dataset. Observing the fact that encouraging inter-class separability is important when applying angular based embedding learning, we propose an exclusive inter-class regularization as a complement for angular based loss. We verify the effectiveness of these methods for learning a discriminative embedding space on ASV task with several experiments. These methods together, we manage to achieve an impressive result with 16.5% improvement on equal error rate (EER) and 18.2% improvement on minimum detection cost function comparing with baseline softmax systems.
Figures
Reference graph
Works this paper leans on
-
[1]
Speaker verification using adapted gaussian mixture models,
D. A. Reynolds, T. F. Quatieri, and R. B. Dunn, “Speaker verification using adapted gaussian mixture models,” Digital signal processing , vol. 10, no. 1-3, pp. 19–41, 2000
work page 2000
-
[2]
A study of interspeaker variability in speaker verification,
P. Kenny, P. Ouellet, N. Dehak, V . Gupta, and P. Dumouchel, “A study of interspeaker variability in speaker verification,” IEEE Transactions on Audio, Speech, and Language Processing , vol. 16, no. 5, pp. 980–988, 2008
work page 2008
-
[3]
Front- end factor analysis for speaker verification,
N. Dehak, P. J. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet, “Front- end factor analysis for speaker verification,” IEEE Transactions on Audio, Speech, and Language Processing , vol. 19, no. 4, pp. 788–798, 2010
work page 2010
-
[4]
Probabilistic linear discriminant analysis for inferences about identity,
S. J. Prince and J. H. Elder, “Probabilistic linear discriminant analysis for inferences about identity,” in 2007 IEEE 11th International Conference on Computer Vision . IEEE, 2007, pp. 1–8
work page 2007
-
[5]
Analysis of i-vector length normalization in speaker recognition systems,
D. Garcia-Romero and C. Y . Espy-Wilson, “Analysis of i-vector length normalization in speaker recognition systems,” in Twelfth annual con- ference of the international speech communication association , 2011
work page 2011
-
[6]
Deep neural network embeddings for text-independent speaker verification
D. Snyder, D. Garcia-Romero, D. Povey, and S. Khudanpur, “Deep neural network embeddings for text-independent speaker verification.” in Interspeech, 2017, pp. 999–1003
work page 2017
-
[7]
X- vectors: Robust dnn embeddings for speaker recognition,
D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur, “X- vectors: Robust dnn embeddings for speaker recognition,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018, pp. 5329–5333
2018
-
[8]
J.-w. Jung, H.-S. Heo, J.-h. Kim, H.-j. Shim, and H.-J. Yu, “Rawnet: Advanced end-to-end deep neural network using raw waveforms for text-independent speaker verification,” arXiv preprint arXiv:1904.08104, 2019
arXiv 1904
Show all 30 references
-
[9]
A complete end-to-end speaker verification system using deep neural networks: From raw signals to verification result,
J.-W. Jung, H.-S. Heo, I.-H. Yang, H.-J. Shim, and H.-J. Yu, “A complete end-to-end speaker verification system using deep neural networks: From raw signals to verification result,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE,...
2018
-
[10]
Avoiding speaker overfitting in end-to-end dnns using raw waveform for text-independent speaker verification,
——, “Avoiding speaker overfitting in end-to-end dnns using raw waveform for text-independent speaker verification,” extraction, vol. 8, no. 12, pp. 23–24, 2018
2018
-
[11]
Generalized end-to-end loss for speaker verification,
L. Wan, Q. Wang, A. Papir, and I. L. Moreno, “Generalized end-to-end loss for speaker verification,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2018, pp. 4879–4883
2018
-
[12]
Deep neural networks for small footprint text-dependent speaker verification,
E. Variani, X. Lei, E. McDermott, I. L. Moreno, and J. Gonzalez- Dominguez, “Deep neural networks for small footprint text-dependent speaker verification,” in 2014 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP) . IEEE, 2014, pp. 4052– 4056
2014
-
[13]
Facenet: A unified embed- ding for face recognition and clustering,
F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embed- ding for face recognition and clustering,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2015, pp. 815– 823
2015
-
[14]
End-to-end text-independent speaker veri- fication with triplet loss on short utterances
C. Zhang and K. Koishida, “End-to-end text-independent speaker veri- fication with triplet loss on short utterances.” in Interspeech, 2017, pp. 1487–1491
2017
-
[15]
Deep speaker: an end-to-end neural speaker embedding system,
C. Li, X. Ma, B. Jiang, X. Li, X. Zhang, X. Liu, Y . Cao, A. Kannan, and Z. Zhu, “Deep speaker: an end-to-end neural speaker embedding system,” arXiv preprint arXiv:1705.02304 , 2017
2017 arXiv
-
[16]
Sphereface: Deep hypersphere embedding for face recognition,
W. Liu, Y . Wen, Z. Yu, M. Li, B. Raj, and L. Song, “Sphereface: Deep hypersphere embedding for face recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 212–220
2017
-
[17]
Cosface: Large margin cosine loss for deep face recognition,
H. Wang, Y . Wang, Z. Zhou, X. Ji, D. Gong, J. Zhou, Z. Li, and W. Liu, “Cosface: Large margin cosine loss for deep face recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 5265–5274
2018
-
[18]
Arcface: Additive angular margin loss for deep face recognition,
J. Deng, J. Guo, N. Xue, and S. Zafeiriou, “Arcface: Additive angular margin loss for deep face recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2019, pp. 4690–4699
2019
-
[19]
Additive margin softmax for face verification,
F. Wang, J. Cheng, W. Liu, and H. Liu, “Additive margin softmax for face verification,” IEEE Signal Processing Letters , vol. 25, no. 7, pp. 926–930, 2018
2018
-
[20]
Triplet loss based cosine similarity metric learning for text-independent speaker recognition,
S. Novoselov, V . Shchemelinin, A. Shulipa, A. Kozlov, and I. Kremnev, “Triplet loss based cosine similarity metric learning for text-independent speaker recognition,” Proc. Interspeech 2018 , pp. 2242–2246, 2018
2018
-
[21]
Angular softmax for short-duration text- independent speaker verification,
Z. Huang, S. Wang, and K. Yu, “Angular softmax for short-duration text- independent speaker verification,” Proc. Interspeech, Hyderabad , 2018
2018
-
[22]
Large margin softmax loss for speaker verification,
Y . Liu, L. He, and J. Liu, “Large margin softmax loss for speaker verification,” arXiv preprint arXiv:1904.03479 , 2019
1904 arXiv
-
[23]
Angular softmax loss for end-to-end speaker verification,
Y . Li, F. Gao, Z. Ou, and J. Sun, “Angular softmax loss for end-to-end speaker verification,” arXiv preprint arXiv:1806.03464 , 2018
2018 arXiv
-
[24]
Regularface: Deep face recognition via exclusive regularization,
K. Zhao, J. Xu, and M.-M. Cheng, “Regularface: Deep face recognition via exclusive regularization,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2019, pp. 1136–1144
2019
-
[25]
Learn- ing towards minimum hyperspherical energy,
W. Liu, R. Lin, Z. Liu, L. Liu, Z. Yu, B. Dai, and L. Song, “Learn- ing towards minimum hyperspherical energy,” in Advances in Neural Information Processing Systems , 2018, pp. 6222–6233
2018
-
[26]
Deep hyperspherical learning,
W. Liu, Y .-M. Zhang, X. Li, Z. Yu, B. Dai, T. Zhao, and L. Song, “Deep hyperspherical learning,” in Advances in neural information processing systems, 2017, pp. 3950–3960
2017
-
[27]
Uniformface: Learning deep equidis- tributed representation for face recognition,
Y . Duan, J. Lu, and J. Zhou, “Uniformface: Learning deep equidis- tributed representation for face recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2019, pp. 3415–3424
2019
-
[28]
V oxceleb: a large-scale speaker identification dataset,
A. Nagrani, J. S. Chung, and A. Zisserman, “V oxceleb: a large-scale speaker identification dataset,” arXiv preprint arXiv:1706.08612 , 2017
2017 arXiv
-
[29]
V oxceleb2: Deep speaker recognition,
J. S. Chung, A. Nagrani, and A. Zisserman, “V oxceleb2: Deep speaker recognition,” arXiv preprint arXiv:1806.05622 , 2018
2018 arXiv
-
[30]
Identity mappings in deep residual networks,
K. He, X. Zhang, S. Ren, and J. Sun, “Identity mappings in deep residual networks,” in European conference on computer vision . Springer, 2016, pp. 630–645
2016
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.