REVIEW 3 major objections 4 minor 42 references
Navigating Label Ambiguity for Facial Expression Recognition in the Wild
T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper claims that a single per-sample weighting mechanism can handle both noisy labels and class imbalance in facial expression recognition, and reports state-of-the-art results on RAF-DB and AffectNet.
desk verdict A solid, incremental FER training method whose SOTA claims rest on small single-run margins and hand-tuned Gaussian hyperparameters that need a sensitivity study. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the Noise-aware Adaptive Weighting (NAW) kernel: a multivariate Gaussian $w^*(\mathbf{p}|\mu,\Sigma)$ evaluated on the pair $\mathbf{p}=(p^{GT}, p^{NN})$ and multiplied into the cross-entropy loss as $(1+w^*)\,L_{CE}$. Separate centers $\mu_t=[0.5,0.5]^T$ for correct predictions and $\mu_f=[0.3,0.15]^T$ for incorrect predictions, with covariance shapes set by axis ratios 2:1 and 6:1 and a scheduler $CS(e,E)=1-\exp(-10e/E)$, determine which samples are treated as ambiguous versus noisy. This kernel is what lets the method emphasize minority-class ambiguous samples late in training; the consistency regularizer (Jensen-Shannon divergence between the logits of original and flipped images) supports it by making the scores on which the kernel depends more stable.
What would settle it
On FERPlus, each image has multiple annotator labels, so genuine ambiguity can be measured directly as the entropy of the label distribution; if the weights NLA assigns are not highest for the high-entropy (genuinely ambiguous) images, or if the reported gains vanish when the Gaussian constants are varied across datasets, the ambiguity-tracking premise is falsified.
Extended reading notes
Core claim
NLA's central proposal is that label ambiguity can be used as a training signal rather than treated as an obstacle. At every iteration the model extracts two numbers from its current softmax output for each sample: $p^{GT}$, the probability of the annotated class, and $p^{NN}$, the probability of the highest-scoring other class. The Noise-aware Adaptive Weighting (NAW) component places these two numbers into a multivariate Gaussian kernel, with one kernel for samples the model currently predicts correctly and another for samples it predicts incorrectly; the kernels' means and covariance shapes are chosen so that the up-weighted samples are the ambiguous ones, while confident and clearly noisy samples receive low weight. A covariance scheduler changes the Gaussian contour from isotropic to elongated over training, shifting emphasis toward ambiguous minority-class samples in later epochs, and a Jensen-Shannon consistency loss between original and flipped images stabilizes the score estimates. The reported consequence is that minority classes such as disgust, fear, and anger improve substantially, reaching over 70% accuracy on RAF-DB for the first time while majority-class accuracy is not sacrificed, yielding the best overall and mean accuracy on two in-the-wild benchmarks.
Load-bearing premise
The entire method rests on the assumption that a fixed, hand-chosen Gaussian formula over two prediction scores separates clean, ambiguous, and noisy samples correctly in every dataset and at every training stage; if those constants only reflect patterns seen in RAF-DB, the reported gains will not transfer.
Editorial extensions
If this is right
- If NLA's results hold, in-the-wild FER can be improved on both overall and mean accuracy by per-sample loss reweighting alone, without separate noise-cleaning or resampling stages.
- The largest gains land in the rarest expressions (disgust, fear, anger), so class-balanced evaluation becomes the right way to compare FER methods, not just overall accuracy.
- The reported robustness under 10–30% label noise and imbalance factors of 50–150 suggests the same loss could be applied directly to newly collected, uncurated face data.
- The ablation study indicates the weighting and the consistency regularizer reinforce each other; removing either one reduces both overall and mean accuracy, so the gains are not attributable to the Gaussian weighting alone.
Reading between the lines
- An extension the paper leaves untested is replacing the fixed Gaussian parameters with a learned estimator of ambiguity; if such an estimator adapts across datasets, the method would shed its main hand-tuned constants.
- Because NAW uses only the model's own softmax scores, it should transfer to other long-tailed classification tasks with noisy labels; the paper's conclusion gestures at this but provides no evidence outside FER.
- The ablation does not separate how much of the gain comes from the Gaussian shape versus the consistency regularizer; a plausible reading is that consistency stabilizes the scores the weights depend on, so a simpler weighting rule plus stronger augmentation could match NLA's results.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes NLA, a framework for in-the-wild facial expression recognition that jointly addresses label ambiguity, label noise, and class imbalance. NLA combines a Noise-aware Adaptive Weighting (NAW) scheme, which reweights the cross-entropy loss via a multivariate Gaussian kernel over the prediction scores for the ground-truth class and the nearest negative class, with a Jensen-Shannon consistency regularization between original and horizontally flipped images. Experiments on RAF-DB, AffectNet, and FERPlus compare NLA with prior methods, report robustness under 10–30% label noise and imbalance factors 50–150, and include ablations and visualizations. The central claims are that NLA surpasses the state of the art in overall and mean accuracy and is the first framework to handle both ambiguity-related noise and class imbalance.
Significance. If the results hold, NLA would be a practically useful contribution because it is a single-backbone method that improves minority-class accuracy without sacrificing majority-class accuracy, and the noise/imbalance experiments (Tables 3–4) report five-seed means with standard deviations, which is a strength. The ablation study (Table 5) indicates that both NAW-CE and JSD consistency contribute to the final accuracy, and the paper provides qualitative evidence in Figs. 5–7. However, the load-bearing parts of the method—the hand-specified Gaussian parameters in Eq. (4) and the exact form of the consistency loss in Eq. (7)—are not yet established with enough rigor, and the headline SOTA margins in Tables 1–2 are small. The central idea is plausible and not circular, since gains are measured on held-out test sets, but the evidence for universal transfer across datasets and noise/imbalance levels is incomplete.
major comments (3)
- [Method, Eq. (4); Implementation Details] The NAW weighting in Eq. (4) depends on several parameters that the paper says are 'experimentally set': μ_t=[0.5,0.5], μ_f=[0.3,0.15], σ_11=0.8, axis ratios 2:1 and 6:1, scheduler constant 10, and λ=0.5. No sensitivity analysis is provided, and the paper does not justify why this fixed kernel transfers across RAF-DB, AffectNet, FERPlus, different noise levels, and different imbalance factors. This is load-bearing because Eq. (4) is the core ambiguity estimator, and the overall gains over MEK in Tables 1–2 are only 0.16 on RAF-DB and 1.33 on AffectNet, margins that could be explained by hyperparameter tuning on the target benchmark. I request a sensitivity study over these parameters (or an argument, ideally with evidence, that the chosen values are not dataset-specific).
- [Eq. (7), Consistency Regularization] The consistency loss is written as D_KL(z_i || (z_i+z'_i)/2) + D_KL(z'_i || (z_i+z'_i)/2), where z_i and z'_i are described as 'output logits'. Kullback-Leibler divergence and Jensen-Shannon divergence are defined for probability distributions, not raw logits, so Eq. (7) as written is undefined. If the implementation applies a softmax before computing the divergence, the paper must state this and write the equation accordingly; if it truly uses logits, the formula is not a valid KL/JSD and needs correction. Since Table 5 shows that the JSD term contributes to the final performance (settings (e) vs. (f)), this is not a purely cosmetic issue.
- [Tables 1–2, Comparison with Existing Methods] The main SOTA comparisons on RAF-DB and AffectNet report single runs with no standard deviation or repeated-seed statistics, unlike Tables 3–4 which do provide five-seed error bars. Given that the margins over the previous best method are small (0.16 overall on RAF-DB and 1.33 on AffectNet), the claim that NLA 'surpasses' the state of the art is not yet statistically supported. Please report mean and standard deviation over multiple seeds for the main benchmark tables, or at least clarify the run-to-run variability.
minor comments (4)
- [Eq. (7)] If the intended quantity is the standard Jensen-Shannon divergence, the formula is missing the factor 1/2 in front of each KL term; either define the loss as the unnormalized sum explicitly or use the standard JSD definition.
- [Related Work / References] The reference list contains duplicate entries for Cui et al. (2019a) and Cui et al. (2019b), which are the same paper; please merge or disambiguate them.
- [Implementation Details] The text states that the best performance is observed at epoch 40 out of a maximum of 60 epochs; please specify whether model selection is performed on a validation split or on the test set, as this affects the interpretation of all reported numbers.
- [Table 5] The column header for the ablation table is difficult to parse; the table would be clearer if the components were listed vertically and each setting indicated which components are enabled.
Circularity Check
No significant circularity; the NAW weighting is a training heuristic evaluated on held-out test sets, not a prediction derived from its own inputs.
full rationale
The paper's central derivation, the NAW weighting in Eqs. (3)-(6) and the total loss in Eq. (8), is a training heuristic whose Gaussian parameters (mu_t, mu_f, sigma_11, axis ratios, scheduler constant) are manually preset and described as 'experimentally set' rather than fitted to the target test accuracies. The claimed gains are measured on held-out test sets in Tables 1-4, so the reported results are not equal to the inputs by construction. The ambiguity measure is an operational definition based on the model's current prediction scores, not a theorem that assumes its own conclusion; the model does not directly optimize per-class test accuracy. There are no load-bearing self-citations: the paper builds on externally published baselines (MEK, EAC, RUL, LA-Net) and standard loss components, and the consistency regularization follows JoCoR-style alignment without invoking an unverified uniqueness claim. Concerns about sensitivity of the hand-set Gaussian constants and the KL-on-logits notation in Eq. (7) are correctness or reproducibility issues, not circularity. Therefore no significant circularity is found.
Assumptions & free parameters
free parameters (5)
- mu_t (true-case Gaussian mean) =
[0.5, 0.5]
- mu_f (false-case Gaussian mean) =
[0.3, 0.15]
- Sigma shape (sigma_11 and axis ratios) =
sigma_11=0.8, axis ratios 2:1 (true) and 6:1 (false)
- Covariance scheduler strength =
10 in CS(e,E)=1-exp(-10*e/E)
- lambda (loss balance) =
0.5
assumptions (5)
- standard math Softmax probabilities and cross-entropy are the correct base model and loss.
- domain assumption Horizontal flipping preserves the facial expression label and latent distribution.
- ad hoc to paper The pair (p_GT, p_NN) at intermediate training stages is a reliable indicator of sample ambiguity and noise.
- ad hoc to paper Ambiguous and noisy samples concentrate in the minority classes in wild FER datasets.
- ad hoc to paper KL and JSD divergence computed on the outputs is a valid distribution alignment term.
Cite this review
Pith. "Pith review of Navigating Label Ambiguity for Facial Expression Recognition in the Wild." pith.science (2026). https://pith.science/paper/GY7SEU7I
@misc{pith2026250209993,
author = {Pith},
title = {Pith review of: Navigating Label Ambiguity for Facial Expression Recognition in the Wild},
year = {2026},
howpublished = {\url{https://pith.science/paper/GY7SEU7I}},
note = {Machine review of arXiv:2502.09993}
}
read the original abstract
Facial expression recognition (FER) remains a challenging task due to label ambiguity caused by the subjective nature of facial expressions and noisy samples. Additionally, class imbalance, which is common in real-world datasets, further complicates FER. Although many studies have shown impressive improvements, they typically address only one of these issues, leading to suboptimal results. To tackle both challenges simultaneously, we propose a novel framework called Navigating Label Ambiguity (NLA), which is robust under real-world conditions. The motivation behind NLA is that dynamically estimating and emphasizing ambiguous samples at each iteration helps mitigate noise and class imbalance by reducing the model's bias toward majority classes. To achieve this, NLA consists of two main components: Noise-aware Adaptive Weighting (NAW) and consistency regularization. Specifically, NAW adaptively assigns higher importance to ambiguous samples and lower importance to noisy ones, based on the correlation between the intermediate prediction scores for the ground truth and the nearest negative. Moreover, we incorporate a regularization term to ensure consistent latent distributions. Consequently, NLA enables the model to progressively focus on more challenging ambiguous samples, which primarily belong to the minority class, in the later stages of training. Extensive experiments demonstrate that NLA outperforms existing methods in both overall and mean accuracy, confirming its robustness against noise and class imbalance. To the best of our knowledge, this is the first framework to address both problems simultaneously.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
S.; Maharaj, T.; Fischer, A.; Courville, A.; Bengio, Y.; et al
Arpit, D.; Jastrz e bski, S.; Ballas, N.; Krueger, D.; Bengio, E.; Kanwal, M. S.; Maharaj, T.; Fischer, A.; Courville, A.; Bengio, Y.; et al. 2017. A closer look at memorization in deep networks. In International conference on machine learning, 233--242. PMLR
work page 2017
-
[2]
Barsoum, E.; Zhang, C.; Ferrer, C. C.; and Zhang, Z. 2016. Training deep networks for facial expression recognition with crowd-sourced label distribution. In Proceedings of the 18th ACM international conference on multimodal interaction, 279--283
work page 2016
-
[3]
Cao, K.; Wei, C.; Gaidon, A.; Arechiga, N.; and Ma, T. 2019. Learning imbalanced datasets with label-distribution-aware margin loss. Advances in neural information processing systems, 32
work page 2019
-
[4]
Cui, Y.; Jia, M.; Lin, T.-Y.; Song, Y.; and Belongie, S. 2019 a . Class-balanced loss based on effective number of samples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9268--9277
work page 2019
-
[5]
Cui, Y.; Jia, M.; Lin, T.-Y.; Song, Y.; and Belongie, S. 2019 b . Class-balanced loss based on effective number of samples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9268--9277
work page 2019
-
[6]
Ghosh, A.; Kumar, H.; and Sastry, P. S. 2017. Robust loss functions under label noise for deep neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 31
work page 2017
-
[7]
Goodfellow, I. J.; Erhan, D.; Carrier, P. L.; Courville, A.; Mirza, M.; Hamner, B.; Cukierski, W.; Tang, Y.; Thaler, D.; Lee, D.-H.; et al. 2013. Challenges in representation learning: A report on three machine learning contests. In Neural Information Processing: 20th International Conference, ICONIP 2013, Daegu, Korea, November 3-7, 2013. Proceedings, Pa...
work page 2013
-
[8]
Guo, Y.; Zhang, L.; Hu, Y.; He, X.; and Gao, J. 2016. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In Computer Vision--ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14, 87--102. Springer
work page 2016
Show all 42 references
-
[9]
Han, B.; Yao, Q.; Yu, X.; Niu, G.; Xu, M.; Hu, W.; Tsang, I.; and Sugiyama, M. 2018. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems, 31
2018
-
[10]
He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770--778
2016
-
[11]
Jiang, L.; Zhou, Z.; Leung, T.; Li, L.-J.; and Fei-Fei, L. 2018. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In International conference on machine learning, 2304--2313. PMLR
2018
-
[12]
Kang, B.; Xie, S.; Rohrbach, M.; Yan, Z.; Gordo, A.; Feng, J.; and Kalantidis, Y. 2019. Decoupling representation and classifier for long-tailed recognition. arXiv preprint arXiv:1910.09217
2019 arXiv
-
[13]
P.; and Ba, J
Kingma, D. P.; and Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980
2014 arXiv
-
[14]
Li, H.; Wang, N.; Ding, X.; Yang, X.; and Gao, X. 2021. Adaptively learning facial expression representation via cf labels and distillation. IEEE Transactions on Image Processing, 30: 2016--2028
2021
-
[15]
Li, S.; Deng, W.; and Du, J. 2017. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2852--2861
2017
-
[16]
Li, Z.; and Arora, S. 2019. An exponential learning rate schedule for deep learning. arXiv preprint arXiv:1910.07454
2019 arXiv
-
[17]
Lin, T.-Y.; Goyal, P.; Girshick, R.; He, K.; and Doll \'a r, P. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, 2980--2988
2017
-
[18]
when to update
Malach, E.; and Shalev-Shwartz, S. 2017. Decoupling" when to update" from" how to update". Advances in neural information processing systems, 30
2017
-
[19]
Mao, J.; Xu, R.; Yin, X.; Chang, Y.; Nie, B.; and Huang, A. 2023. POSTER++: A simpler and stronger facial expression recognition network. arXiv preprint arXiv:2301.12149
2023 arXiv
-
[20]
Mollahosseini, A.; Hasani, B.; and Mahoor, M. H. 2017. Affectnet: A database for facial expression, valence, and arousal computing in the wild. IEEE Transactions on Affective Computing, 10(1): 18--31
2017
-
[21]
Ren, J.; Yu, C.; Ma, X.; Zhao, H.; Yi, S.; et al. 2020. Balanced meta-softmax for long-tailed visual recognition. Advances in neural information processing systems, 33: 4175--4186
2020
-
[22]
She, J.; Hu, Y.; Shi, H.; Wang, J.; Shen, Q.; and Mei, T. 2021. Dive into ambiguity: Latent distribution mining and pairwise uncertainty estimation for facial expression recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 6248--6257
2021
-
[23]
Wang, K.; Peng, X.; Yang, J.; Lu, S.; and Qiao, Y. 2020 a . Suppressing uncertainties for large-scale facial expression recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 6897--6906
2020
-
[24]
Wang, K.; Peng, X.; Yang, J.; Meng, D.; and Qiao, Y. 2020 b . Region attention networks for pose and occlusion robust facial expression recognition. IEEE Transactions on Image Processing, 29: 4057--4069
2020
-
[25]
Wang, X.; Bo, L.; and Fuxin, L. 2019. Adaptive wing loss for robust face alignment via heatmap regression. In Proceedings of the IEEE/CVF international conference on computer vision, 6971--6981
2019
-
[26]
Wang, Y.; Gan, W.; Yang, J.; Wu, W.; and Yan, J. 2019. Dynamic curriculum learning for imbalanced data classification. In Proceedings of the IEEE/CVF international conference on computer vision, 5017--5026
2019
-
[27]
Wei, H.; Feng, L.; Chen, X.; and An, B. 2020. Combating noisy labels by agreement: A joint training method with co-regularization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 13726--13735
2020
-
[28]
Wu, Z.; and Cui, J. 2023. LA-Net: Landmark-Aware Learning for Reliable Facial Expression Recognition under Label Noise. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 20698--20707
2023
-
[29]
Xue, F.; Wang, Q.; and Guo, G. 2021. Transfer: Learning relation-aware facial expression representations with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 3601--3610
2021
-
[30]
Xue, F.; Wang, Q.; Tan, Z.; Ma, Z.; and Guo, G. 2022. Vision transformer with attentive pooling for robust facial expression recognition. IEEE Transactions on Affective Computing
2022
-
[31]
Yu, X.; Han, B.; Yao, J.; Niu, G.; Tsang, I.; and Sugiyama, M. 2019. How does disagreement help generalization against label corruption? In International conference on machine learning, 7164--7173. PMLR
2019
-
[32]
Zeng, D.; Lin, Z.; Yan, X.; Liu, Y.; Wang, F.; and Tang, B. 2022. Face2exp: Combating data biases for facial expression recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 20291--20300
2022
-
[33]
Zhang, Y.; Li, Y.; Liu, X.; Deng, W.; et al. 2024. Leave No Stone Unturned: Mine Extra Knowledge for Imbalanced Facial Expression Recognition. Advances in Neural Information Processing Systems, 36
2024
-
[34]
Zhang, Y.; Wang, C.; and Deng, W. 2021. Relative uncertainty learning for facial expression recognition. Advances in Neural Information Processing Systems, 34: 17616--17627
2021
-
[35]
Zhang, Y.; Wang, C.; Ling, X.; and Deng, W. 2022 a . Learn from all: Erasing attention consistency for noisy label facial expression recognition. In European Conference on Computer Vision, 418--434. Springer
2022
-
[36]
Zhang, Z.; Sun, X.; Li, J.; and Wang, M. 2022 b . MAN: Mining ambiguity and noise for facial expression recognition in the wild. Pattern Recognition Letters, 164: 23--29
2022
-
[37]
Zhao, Z.; Liu, Q.; and Zhou, F. 2021. Robust lightweight facial expression recognition network with label distribution training. In Proceedings of the AAAI conference on artificial intelligence, volume 35, 3510--3519
2021
-
[38]
Zheng, C.; Mendieta, M.; and Chen, C. 2023. Poster: A pyramid cross-fusion transformer network for facial expression recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 3146--3155
2023
-
[39]
Zhou, B.; Cui, Q.; Wei, X.-S.; and Chen, Z.-M. 2020. Bbn: Bilateral-branch network with cumulative learning for long-tailed visual recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9719--9728
2020
-
[40]
Zhou, B.; Khosla, A.; Lapedriza, A.; Oliva, A.; and Torralba, A. 2016. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2921--2929
2016
-
[41]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
-
[42]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.