Pith. sign in

REVIEW 4 major objections 9 minor 80 references

Enhancing Ambiguous Dynamic Facial Expression Recognition with Soft Label-based Data Augmentation

T0 review · 4 major / 9 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Mixing video frames and soft labels lifts emotion recognition beyond the state of the art.

desk verdict The MIDAS method itself is from WACV 2024; the new value is the FERV39k-Plus dataset and the wider architecture sweep, but the claimed superiority over SOTA lacks variance estimates and partially reduces to soft-label-only gains. read the letter →

arxiv 2506.20867 v1 pith:4J34NUND submitted 2025-06-25 cs.CV

Ryosuke Kawamura , Hideaki Hayashi , Shunsuke Otake , Noriko Takemura , Hajime Nagahara This is my paper
classification cs.CV
keywords dynamicfacialexpressionrecognitiondataaugmentationsoftlabelsmixupvicinalriskminimizationambiguousexpressionsDFEWdatasetFERV39k-Plus
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a data augmentation method called MIDAS improves dynamic facial expression recognition by convexly mixing pairs of video frames together with their soft labels, which are probability distributions over emotion classes rather than single one-hot labels. On the DFEW benchmark, MAE-DFER trained with MIDAS reaches a UAR of 63.68 and a WAR of 74.84, surpassing the same model trained on hard labels (60.76 and 72.76) as well as previously published state-of-the-art results. The authors also construct FERV39k-Plus, a soft-labeled version of an existing video expression dataset, and report consistent gains across three model architectures and in cross-dataset transfer. The practical interest is that ambiguous expressions are common in real-world video, and this approach turns annotator disagreement into usable training signal instead of discarding it.

What carries the argument

The central object is the frame-wise convex-mixing rule $\tilde{x}^{(t)} = \lambda x_i^{(t)} + (1-\lambda) x_j^{(t)}$ and $\tilde{y} = \lambda y_i + (1-\lambda) y_j$, where the mixing weight $\lambda$ is drawn from a $\beta$ distribution with $\alpha = 0.8$, and the combined label is renormalized with a softmax. It carries the argument by generating virtual video clips whose soft labels are the same interpolation as their pixels, and it is justified as minimizing vicinal risk under the condition that the true hard label is unknown.

What would settle it

Take a held-out set of video pairs with very different expressions, generate blended clips across a range of mixing weights, and check whether a MIDAS-trained model's predicted emotion distribution tracks the interpolated soft label; a systematic mismatch as the weight varies would falsify the linearity assumption.

Watch

Extended reading notes

Core claim

MIDAS treats an ambiguous facial expression as a mixture of emotions and augments training data by selecting two video clips, sampling a mixing weight $\lambda$ from a $\beta$ distribution, interpolating every corresponding frame as $\tilde{x}^{(t)} = \lambda x_i^{(t)} + (1-\lambda) x_j^{(t)}$, and interpolating the two soft labels with the same weight: $\tilde{y} = \lambda y_i + (1-\lambda) y_j$, followed by softmax normalization of $\tilde{y}$. The paper's central claim is that these mixed videos with mixed soft labels are better training data than either the original hard-labeled videos or the original soft-labeled videos alone, and it justifies this through vicinal risk minimization: since the true hard label is unknown and only annotator-vote averages are available, MIDAS minimizes a vicinal risk whose vicinity distribution is a soft-label variant of the one used by original mixup. The experiments on DFEW and FERV39k-Plus show that MAE-DFER, TSM, and VideoMAE all improve when trained with MIDAS, with the largest per-class gains on rare emotions such as Disgust and Fear. The conclusion is that soft labels and mixing are synergistic rather than redundant.

Load-bearing premise

MIDAS assumes that blending two facial-expression videos frame by frame in pixel space yields a training video whose correct emotion label is exactly the weighted average of the two source soft labels.

Editorial extensions

If this is right

  • On DFEW, MIDAS raises UAR from 60.76 to 63.68 and WAR from 72.76 to 74.84 compared with MAE-DFER trained on hard labels.
  • MIDAS applied to hard labels alone yields much smaller gains (UAR +0.65 on DFEW), so the effect depends on combining mixing with soft labels.
  • The improvement holds across TSM, VideoMAE, and MAE-DFER on both DFEW and FERV39k-Plus, indicating the benefit is not architecture-specific.
  • In cross-dataset evaluation, MIDAS improves WAR for models trained on DFEW and tested on FERV39k-Plus, and improves both UAR and WAR in the reverse direction.
  • Per-class results show the largest gains on rare classes: Disgust accuracy on DFEW goes from 11.72 to 20.00 and Fear from 36.14 to 40.02.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same recipe should transfer to other tasks with distributional labels, such as medical image grading or speech emotion, wherever annotator disagreement is available; the paper itself only tests DFER.
  • Editorial inference: the per-class pattern suggests MIDAS acts partly as a rare-class regularizer, so a testable consequence is that gains will shrink if the training set is artificially balanced before mixing.
  • Editorial inference: since mixing happens in pixel space, the method cannot synthesize new temporal dynamics; blending clips with different expression onsets may teach averaged dynamics, which may explain why Surprise accuracy in FERV39k-Plus remains low.
  • Editorial inference: the softmax normalization of mixed labels may contribute a label-smoothing effect, so a controlled comparison against standard label smoothing would separate interpolation from normalization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 9 minor

Summary. The paper proposes MIDAS, a data augmentation method that extends mixup from hard-label images to soft-labeled video clips for dynamic facial expression recognition (DFER). MIDAS convexly combines pairs of video frames and their corresponding soft emotion labels, with a mixing coefficient sampled from a beta distribution. The authors also construct FERV39k-Plus, a new soft-labeled version of FERV39k obtained by crowd-sourced annotations, and evaluate MIDAS with MAE-DFER, TSM, and VideoMAE on both DFEW and FERV39k-Plus. The reported results show UAR/WAR improvements over hard-label training on DFEW, mixed results on FERV39k-Plus, and additional analyses of cross-dataset generalization, alpha sensitivity, and the effect of ambiguous data.

Significance. If the reported gains are robust, the paper makes a useful contribution: it directly targets the ambiguity that is pervasive in in-the-wild DFER, proposes a simple and architecture-agnostic augmentation, and introduces a new soft-labeled dataset that could support future work. The cross-architecture and cross-dataset evaluations are a strength, and the analysis of coexisting emotions provides interpretive value. However, the central empirical claim is not yet established with statistical confidence: all comparisons are single-run point estimates with no error bars, repeated seeds, or significance tests, and several table entries are internally inconsistent. The theoretical justification in Sec. III-B is also largely definitional rather than predictive. The contribution is potentially valuable, but the evidence as presented is not yet at the bar required for acceptance.

major comments (4)
  1. [Sec. IV-C, Tables III-V] The central claim that MIDAS outperforms the state-of-the-art method trained on the original data rests on single-run point estimates with no variance information or significance testing. On DFEW (Table III), the gains of MAE-DFER+MIDAS over hard-label MAE-DFER are UAR +2.92 and WAR +2.08, but Table V shows that soft labels alone already provide UAR +1.88 and WAR +1.76, leaving only +1.04 UAR and +0.32 WAR attributable to the mixing mechanism. On FERV39k-Plus (Table IV), WAR is exactly equal (68.65) for hard-label MAE-DFER and MIDAS, and the UAR gain (+2.00) is driven by a large Fear increase (13.04 to 20.65) alongside a Disgust collapse (5.76 to 0.36). Without repeated-seed means, standard deviations, or a significance test, the reader cannot determine whether these differences are training noise. This is the load-bearing issue for the paper's headline claim.
  2. [Sec. IV-B2 and Sec. V-F, Table IX] The main experiments fix the beta-distribution parameter at alpha=0.8, but Table IX reports that alpha=0.3 gives the best UAR (64.21) and alpha=0.4 gives the best WAR (74.89) among the tested values. The paper does not state whether Table IX is computed on the validation split or the test split, and it does not explain why alpha=0.8 is used for the headline numbers. If Table IX is a validation-based selection, the headline results use a suboptimal hyperparameter under the paper's own model-selection protocol; if it is a test-based exploration, the comparison is inconsistent with the stated emphasis on validation-based selection. The authors should clarify the selection procedure and report the main results using the selected alpha value, or justify the fixed choice.
  3. [Sec. IV-A2, Tables IV-VI] The construction of FERV39k-Plus removes all video clips without a single most-voted class, i.e., clips with tied top votes are discarded. Since ties are among the most ambiguous samples, this biases the new dataset away from the very phenomenon the method targets and makes it incomplete as a soft-labeled version of FERV39k. In addition, the hard-label baseline numbers for the same condition are inconsistent across tables: MAE-DFER hard-label UAR on FERV39k-Plus is reported as 43.28 in Table IV but 43.15 in Tables V and VI, and VideoMAE hard-label WAR is 65.21 in Table IV but 65.12 in Table V. These discrepancies undermine confidence in the reported comparisons and must be corrected.
  4. [Sec. III-B, Eqs. (2)-(3)] The vicinal-risk derivation in Sec. III-B is definitional: Eq. (8) restates the MIDAS sampling procedure as a distribution, and the statement that MIDAS minimizes the empirical vicinal risk under that distribution is true by construction. The derivation does not provide a testable justification for the core assumption that linear interpolation of video frames in pixel space, paired with linear interpolation of soft labels, yields valid training samples whose blended labels represent the actual expression distribution. This assumption is load-bearing for the method's validity. The authors should either provide a theoretically grounded argument or include an experiment that directly tests the linearity assumption, for example by comparing pixel-space mixing with feature-space mixing or by evaluating whether the generated mixed samples are semantically plausible.
minor comments (9)
  1. [Sec. VI] The conclusion states that MIDAS "was evaluated using only the DFEW dataset," but the paper also reports extensive experiments on FERV39k-Plus; this sentence should be corrected to avoid contradicting the paper's own results.
  2. [Sec. V-C] The text refers to "the FERV39k-S test set," which appears to be a typo for FERV39k-Plus; please correct it.
  3. [Sec. IV-A1] There is a typo in "his dataset includes the class with the highest number of votes" which should read "this dataset includes the class with the highest number of votes."
  4. [Abstract and Sec. IV-C2] The claim of "superior performance" is overstated for FERV39k-Plus, where Table IV shows equal WAR (68.65) for MAE-DFER hard-label and MAE-DFER+MIDAS; the abstract should be qualified accordingly.
  5. [Sec. III-A] The softmax normalization of the mixed soft label is unexplained. A convex combination of two probability vectors is already normalized, so the softmax operation changes the label distribution in a nonlinear way; the paper should state the reason and any temperature parameter used.
  6. [Sec. III-B, Eq. (9)] In Eq. (9), lambda' is defined as lambda * l / S and is therefore a deterministic function of lambda and l, not an independent random variable; writing E_{lambda, lambda'} is misleading. Additionally, the definition of y'_j mixes indices from both i and j, making the expression difficult to interpret. The notation should be clarified.
  7. [Sec. V-F, Table IX] Table IX does not indicate whether the reported UAR/WAR are on the validation split or the test split, and it is unclear whether the same alpha was used across the five DFEW folds or selected per fold. Please state this explicitly.
  8. [Sec. IV-A2] The FERV39k-Plus dataset is described as "will be made publicly available soon," which prevents current verification of the new dataset and the cross-dataset results. The paper should include a release URL or supplement with the annotation statistics and a sample of the annotation interface.
  9. [Sec. V-E, Table VIII] The comparison in Table VIII is not clean: the "mixed expression group" contains data regardless of soft-label values and therefore includes clear examples, so it is a superset of the "clear expression group" rather than a disjoint ambiguous group. The conclusion that ambiguous data improve performance would be better supported by a three-way comparison (clear only, ambiguous only, and full data) with matched class distributions.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor, non-load-bearing circularity: the VRM 'justification' in Sec. III-B rewrites the MIDAS sampling rule as a distribution and then shows MIDAS minimizes risk under that distribution; the central empirical claim rests on external benchmark evaluations.

  1. self definitional [Sec. III-B, Eqs. (8)-(9) and the paragraph following Eq. (9)]
    "PMIDAS(˜xi, ˜yi | xi, qi) = 1/n Σ_j E_λ [δ(˜xi = λxi + (1 − λ)xj, ˜yi = λqi + (1 − λ)qj)] ... Using this expression, Eq. (8) can be written as ... where we defined λ′ = λl/S and y′j = λ/(S−λl) Σ_{s∈W} v_i^(s) + S(1−λ)/(S−λl) qj. ... MIDAS corresponds to minimizing the vicinal risk in a situation where the true hard label yi is unknown."

    Eq. (8)'s vicinity distribution is literally the MIDAS sampling rule from Eqs. (2)-(3): sample λ~β(α,α), set x~=λx_i+(1−λ)x_j and y~=λq_i+(1−λ)q_j. Eq. (9) is then obtained by substituting q_i=(l/S)y_i+(1/S)Σ_{s∈W}v_i^(s) and defining λ'=λl/S, y'_j=λ/(S−λl)Σ_{s∈W}v_i^(s)+S(1−λ)/(S−λl)q_j; with these definitions λ'y_i+(1−λ')y'_j=λq_i+(1−λ)q_j identically. Thus the statement that MIDAS 'corresponds to minimizing the vicinal risk' is a restatement of its own definition, not an independent derivation. The equivalence is also conditional on unobserved quantities (the true label y_i and the set W of wrong votes). Since the empirical tables are external-benchmark comparisons and no hyperparameter is chosen by this derivation, the circularity is minor and non-load-bearing.

full rationale

The paper's central claim—MIDAS improves DFER over the state-of-the-art baseline—is an empirical claim evaluated on the external DFEW benchmark and on the newly annotated FERV39k-Plus dataset; no parameter of the model or the label-generation process is fitted to test outputs. The only place where a derivation reduces to its own definition is Sec. III-B: Eq. (8) defines the vicinity distribution to be exactly the convex mixing rule in Eqs. (2)-(3), and Eq. (9) is an algebraic rewrite (with λ' and y'_j defined accordingly) rather than an independent justification. This does not drive any experimental number. The paper cites its own WACV 2024 paper [5] as the origin of MIDAS, but the method is fully described and re-implemented here, and the comparison baselines are external, so the self-citation is not load-bearing. No fitted parameter is renamed as a prediction, and no external result is imported solely by same-author citation. I therefore find minor circular content (score 2), with the central empirical claim self-contained against external benchmarks.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim rests on one free hyperparameter (alpha), the standard mixup linearity assumption, and an assumed generative model for soft labels from annotator votes. No new physical or theoretical entities are introduced. The FERV39k-Plus dataset is a new artifact but not a theoretical postulation.

free parameters (1)
  • alpha (mixing coefficient beta distribution parameter) = 0.8
    Controls the strength of mixing in Eq (2)-(3). Set to 0.8 in the main experiments; the ablation in Table IX shows UAR is highest at alpha=0.3 and WAR at alpha=0.4, so the chosen value is not the best according to the paper's own sweep.
assumptions (3)
  • domain assumption Linear interpolation in input space corresponds to linear interpolation in label space for facial expression videos.
    MIDAS relies on this mixup assumption to generate valid training pairs; it is invoked in Eq (2) and (3) and not empirically validated for dynamic expressions.
  • domain assumption The soft label qi is the average of S independent one-hot annotator votes and is an unbiased estimate of the true class distribution.
    This model underlies the vicinal risk derivation in Section III-B, specifically Eq (8)-(9), and assumes annotator noise is symmetric and the majority is correct.
  • standard math Vicinal risk minimization is an appropriate objective for DFER with soft labels.
    The paper applies the vicinal risk framework from [48] to justify MIDAS; this is a standard statistical learning assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Ambiguous Dynamic Facial Expression Recognition with Soft Label-based Data Augmentation." pith.science (2026). https://pith.science/paper/4J34NUND

@misc{pith2026250620867,
  author       = {Pith},
  title        = {Pith review of: Enhancing Ambiguous Dynamic Facial Expression Recognition with Soft Label-based Data Augmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4J34NUND}},
  note         = {Machine review of arXiv:2506.20867}
}
read the original abstract

Dynamic facial expression recognition (DFER) is a task that estimates emotions from facial expression video sequences. For practical applications, accurately recognizing ambiguous facial expressions -- frequently encountered in in-the-wild data -- is essential. In this study, we propose MIDAS, a data augmentation method designed to enhance DFER performance for ambiguous facial expression data using soft labels representing probabilities of multiple emotion classes. MIDAS augments training data by convexly combining pairs of video frames and their corresponding emotion class labels. This approach extends mixup to soft-labeled video data, offering a simple yet highly effective method for handling ambiguity in DFER. To evaluate MIDAS, we conducted experiments on both the DFEW dataset and FERV39k-Plus, a newly constructed dataset that assigns soft labels to an existing DFER dataset. The results demonstrate that models trained with MIDAS-augmented data achieve superior performance compared to the state-of-the-art method trained on the original dataset.

Figures

Figures reproduced from arXiv: 2506.20867 by the authors.

Figure 1
Figure 1. Example of an ambiguous facial expression. The [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Outline of the data mixing procedure in MIDAS. In MIDAS, training data are augmented by convexly combining pairs [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. To construct the FERV39k-Plus dataset, annotations [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (6 more)
Figure 3
Figure 3. Figure 3: Examples of a clear facial expression (left) and ambigu [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 4
Figure 4. Figure 4: Examples of a clear facial expression (left) and ambigu [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: The ratio of coexisting emotions for each emotion class in the DFEW dataset [4] and the FERV39k-Plus dataset is [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Confusion matrix of MAE-DFER with MIDAS trained on DFEW dataset [4] and FERV39k-Plus datasets [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Posterior probabilities and ground truth soft label on DFEW dataset [4]. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Posterior probabilities and ground truth soft label on FERV39k-Plus dataset. [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

80 extracted references · 65 canonical work pages

  1. [1]

    Learning expressionlets on spatio-temporal manifold for dynamic facial expression recognition,

    M. Liu, S. Shan, R. Wang, and X. Chen, “Learning expressionlets on spatio-temporal manifold for dynamic facial expression recognition,” in Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 1749–1756. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 11 Frames Happy Sad Neutral Angry Surprise Disgust Fear 0....

    work page 2014

  2. [2]

    Facial expression recogni- tion using encoded dynamic features,

    P. Yang, Q. Liu, X. Cui, and D. N. Metaxas, “Facial expression recogni- tion using encoded dynamic features,” in Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2008, pp. 1–8

    work page 2008

  3. [3]

    Boosting encoded dynamic features for facial expression recognition,

    P. Yang, Q. Liu, and D. N. Metaxas, “Boosting encoded dynamic features for facial expression recognition,” Pattern Recognition Letters, vol. 30, no. 2, pp. 132–139, 2009

    work page 2009

  4. [4]

    Dfew: A large-scale database for recognizing dynamic facial expressions in the wild,

    X. Jiang, Y . Zong, W. Zheng, C. Tang, W. Xia, C. Lu, and J. Liu, “Dfew: A large-scale database for recognizing dynamic facial expressions in the wild,” in Proceedings of the International Conference on Multimedia , 2020, pp. 2881–2889

    work page 2020

  5. [5]

    Midas: Mixing ambiguous data with soft labels for dynamic facial expression recognition,

    R. Kawamura, H. Hayashi, N. Takemura, and H. Nagahara, “Midas: Mixing ambiguous data with soft labels for dynamic facial expression recognition,” in Proceedings of the IEEE/CVF Winter Conference on JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 12 Applications of Computer Vision , 2024, pp. 6552–6562

    work page 2021

  6. [6]

    FERV39k: A large-scale multi-scene dataset for facial expression recognition in videos,

    Y . Wang, Y . Sun, Y . Huang, Z. Liu, S. Gao, W. Zhang, W. Ge, and W. Zhang, “FERV39k: A large-scale multi-scene dataset for facial expression recognition in videos,” in Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) , 2022, pp. 20 922– 20 931

    work page 2022

  7. [7]

    The extended Cohn-Kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression,

    P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, “The extended Cohn-Kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression,” in Proceedings of the Computer Society Conference on Computer Vision and Pattern Recog- nition - Workshops, Jun. 2010, pp. 94–101

    work page 2010

  8. [8]

    Facial expression recognition from near-infrared videos,

    G. Zhao, X. Huang, M. Taini, S. Z. Li, and M. Pietik ¨aInen, “Facial expression recognition from near-infrared videos,” Image and Vision Computing, vol. 29, no. 9, pp. 607–619, 2011

    work page 2011

Show all 80 references
  1. [9]

    Collecting large, richly annotated facial-expression databases from movies,

    A. Dhall, R. Goecke, S. Lucey, and T. Gedeon, “Collecting large, richly annotated facial-expression databases from movies,” IEEE MultiMedia, vol. 19, no. 3, pp. 34–41, Jul. 2012

    2012

  2. [10]

    Mafw: A large-scale, multi-modal, compound affective database for dynamic facial expression recognition in the wild,

    Y . Liu, W. Dai, C. Feng, W. Wang, G. Yin, J. Zeng, and S. Shan, “Mafw: A large-scale, multi-modal, compound affective database for dynamic facial expression recognition in the wild,” in Proceedings of the 30th ACM international conference on multimedia , 2022, pp. 24–32

    2022

  3. [11]

    Peak-piloted deep network for facial expression recognition,

    X. Zhao, X. Liang, L. Liu, T. Li, Y . Han, N. Vasconcelos, and S. Yan, “Peak-piloted deep network for facial expression recognition,” in Proceedings of the European Conference on Computer Vision (ECCV) . Springer, 2016, pp. 425–442

    2016

  4. [12]

    Frame attention networks for facial expression recognition in videos,

    D. Meng, X. Peng, K. Wang, and Y . Qiao, “Frame attention networks for facial expression recognition in videos,” in Proceedings of the International Conference on Image Processing (ICIP) . IEEE, 2019, pp. 3866–3870

    2019

  5. [13]

    Convolu- tional neural networks pretrained on large face recognition datasets for emotion classification from video,

    B. Knyazev, R. Shvetsov, N. Efremova, and A. Kuharenko, “Convolu- tional neural networks pretrained on large face recognition datasets for emotion classification from video,” arXiv preprint arXiv:1711.04598 , 2017

    2017 arXiv

  6. [14]

    Deeper cascaded peak-piloted network for weak expression recognition,

    Z. Yu, Q. Liu, and G. Liu, “Deeper cascaded peak-piloted network for weak expression recognition,” The Visual Computer, vol. 34, no. 12, pp. 1691–1699, 2018

    2018

  7. [15]

    Facial expression recognition by de-expression residue learning,

    H. Yang, U. Ciftci, and L. Yin, “Facial expression recognition by de-expression residue learning,” in Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) , 2018, pp. 2168– 2177

    2018

  8. [16]

    Noisy student training using body language dataset improves facial expression recognition,

    V . Kumar, S. Rao, and L. Yu, “Noisy student training using body language dataset improves facial expression recognition,” inProceedings of the European Conference on Computer Vision (ECCV) . Springer, 2020, pp. 756–773

    2020

  9. [17]

    Multi-objective based spatio-temporal feature representation learning robust to expression intensity variations for facial expression recognition,

    D. H. Kim, W. J. Baddar, J. Jang, and Y . M. Ro, “Multi-objective based spatio-temporal feature representation learning robust to expression intensity variations for facial expression recognition,”IEEE Transactions on Affective Computing , vol. 10, no. 2, pp. 223–236, 2017

    2017

  10. [18]

    Multi-cue fusion for emotion recognition in the wild,

    J. Yan, W. Zheng, Z. Cui, C. Tang, T. Zhang, and Y . Zong, “Multi-cue fusion for emotion recognition in the wild,” Neurocomputing, vol. 309, pp. 27–35, 2018

    2018

  11. [19]

    Stcam: Spatial-temporal and channel attention module for dynamic facial expression recognition,

    W. Chen, D. Zhang, M. Li, and D.-J. Lee, “Stcam: Spatial-temporal and channel attention module for dynamic facial expression recognition,” IEEE Transactions on Affective Computing , 2020

    2020

  12. [20]

    Deep generative- contrastive networks for facial expression recognition,

    Y . Kim, B. Yoo, Y . Kwak, C. Choi, and J. Kim, “Deep generative- contrastive networks for facial expression recognition,” arXiv preprint arXiv:1703.07140, 2017

    2017 arXiv

  13. [21]

    Temporal multimodal fusion for video emotion classification in the wild,

    V . Vielzeuf, S. Pateux, and F. Jurie, “Temporal multimodal fusion for video emotion classification in the wild,” in Proceedings of the International Conference on Multimodal Interaction, 2017, pp. 569–576

    2017

  14. [22]

    Facial expression recognition based on deep convolution long short-term memory networks of double- channel weighted mixture,

    H. Zhang, B. Huang, and G. Tian, “Facial expression recognition based on deep convolution long short-term memory networks of double- channel weighted mixture,” Pattern Recognition Letters , vol. 131, pp. 128–134, 2020

    2020

  15. [23]

    Saanet: Siamese action-units attention network for improving dynamic facial expression recognition,

    D. Liu, X. Ouyang, S. Xu, P. Zhou, K. He, and S. Wen, “Saanet: Siamese action-units attention network for improving dynamic facial expression recognition,” Neurocomputing, vol. 413, pp. 145–157, 2020

    2020

  16. [24]

    Emotion recognition with spatial attention and temporal softmax pooling,

    M. Aminbeidokhti, M. Pedersoli, P. Cardinal, and E. Granger, “Emotion recognition with spatial attention and temporal softmax pooling,” in Proceedings of the International Conference on Image Analysis and Recognition (ICIAR). Springer, 2019, pp. 323–331

    2019

  17. [25]

    Former-dfer: Dynamic facial expression recog- nition transformer,

    Z. Zhao and Q. Liu, “Former-dfer: Dynamic facial expression recog- nition transformer,” in Proceedings of the International Conference on Multimedia, 2021, pp. 1553–1561

    2021

  18. [26]

    Spatio-temporal transformer for dy- namic facial expression recognition in the wild,

    F. Ma, B. Sun, and S. Li, “Spatio-temporal transformer for dy- namic facial expression recognition in the wild,” arXiv preprint arXiv:2205.04749, 2022

    2022 arXiv

  19. [27]

    Dpcnet: Dual path multi-excitation collaborative network for facial expression representation learning in videos,

    Y . Wang, Y . Sun, W. Song, S. Gao, Y . Huang, Z. Chen, W. Ge, and W. Zhang, “Dpcnet: Dual path multi-excitation collaborative network for facial expression representation learning in videos,” in Proceedings of the International Conference on Multimedia , 2022, pp. 101–110

    2022

  20. [28]

    Fru-adapter: Frame recalibration unit adapter for dynamic facial expression recognition,

    M. Her, H. G. Nabi, and J.-H. Han, “Fru-adapter: Frame recalibration unit adapter for dynamic facial expression recognition,” Electronics, vol. 14, no. 5, p. 978, 2025

    2025

  21. [29]

    From static to dynamic: Adapting landmark-aware image models for facial expression recognition in videos,

    Y . Chen, J. Li, S. Shan, M. Wang, and R. Hong, “From static to dynamic: Adapting landmark-aware image models for facial expression recognition in videos,” IEEE Transactions on Affective Computing , pp. 1–15, 2024

    2024

  22. [30]

    Rethink- ing the learning paradigm for dynamic facial expression recognition,

    H. Wang, B. Li, S. Wu, S. Shen, F. Liu, S. Ding, and A. Zhou, “Rethink- ing the learning paradigm for dynamic facial expression recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 17 958–17 968

    2023

  23. [31]

    Mae-dfer: Efficient masked au- toencoder for self-supervised dynamic facial expression recognition,

    L. Sun, Z. Lian, B. Liu, and J. Tao, “Mae-dfer: Efficient masked au- toencoder for self-supervised dynamic facial expression recognition,” in Proceedings of the 31st ACM International Conference on Multimedia , 2023, pp. 6110–6121

    2023

  24. [32]

    Svfap: Self-supervised video facial affect perceiver,

    L. Sun, Z. Lian, K. Wang, Y . He, M. Xu, H. Sun, B. Liu, and J. Tao, “Svfap: Self-supervised video facial affect perceiver,”IEEE Transactions on Affective Computing , 2024

    2024

  25. [33]

    Fe-adapter: Adapting image- based emotion classifiers to videos,

    S. N. Gowda, B. Gao, and D. A. Clifton, “Fe-adapter: Adapting image- based emotion classifiers to videos,” in 2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG) . IEEE, 2024, pp. 1–6

    2024

  26. [34]

    Hicmae: Hierarchical contrastive masked autoencoder for self-supervised audio-visual emotion recogni- tion,

    L. Sun, Z. Lian, B. Liu, and J. Tao, “Hicmae: Hierarchical contrastive masked autoencoder for self-supervised audio-visual emotion recogni- tion,” Information Fusion, vol. 108, p. 102382, 2024

    2024

  27. [35]

    Mma-dfer: Multimodal adaptation of unimodal models for dynamic facial expression recognition in-the-wild,

    K. Chumachenko, A. Iosifidis, and M. Gabbouj, “Mma-dfer: Multimodal adaptation of unimodal models for dynamic facial expression recognition in-the-wild,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 4673–4682

    2024

  28. [36]

    Emoclip: A vision-language method for zero-shot video facial expression recognition,

    N. M. Foteinopoulou and I. Patras, “Emoclip: A vision-language method for zero-shot video facial expression recognition,” in 2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG). IEEE, 2024, pp. 1–10

    2024

  29. [37]

    Finecliper: Multi- modal fine-grained clip for dynamic facial expression recognition with adapters,

    H. Chen, H. Huang, J. Dong, M. Zheng, and D. Shao, “Finecliper: Multi- modal fine-grained clip for dynamic facial expression recognition with adapters,” in Proceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 2301–2310

    2024

  30. [38]

    Emotion distribution recognition from facial expressions,

    Y . Zhou, H. Xue, and X. Geng, “Emotion distribution recognition from facial expressions,” in Proceedings of the International Conference on Multimedia, ser. MM ’15. New York, NY , USA: Association for Computing Machinery, 2015, pp. 1247–1250

    2015

  31. [39]

    Expression recognition for severely demented patients in music reminiscence-therapy,

    A. Dantcheva, P. Bilinski, H. T. Nguyen, J.-C. Broutart, and F. Bre- mond, “Expression recognition for severely demented patients in music reminiscence-therapy,” in Proceedings of the European Signal Process- ing Conference (EUSIPCO) . IEEE, 2017, pp. 783–787

    2017

  32. [40]

    Dive into am- biguity: Latent distribution mining and pairwise uncertainty estimation for facial expression recognition,

    J. She, Y . Hu, H. Shi, J. Wang, Q. Shen, and T. Mei, “Dive into am- biguity: Latent distribution mining and pairwise uncertainty estimation for facial expression recognition,” in Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) . Nashville, TN, ...

    2021

  33. [41]

    Suppressing uncer- tainties for large-scale facial expression recognition,

    K. Wang, X. Peng, J. Yang, S. Lu, and Y . Qiao, “Suppressing uncer- tainties for large-scale facial expression recognition,” in Proceedings of the Conference on Computer Vision and Pattern Recognition , 2020, pp. 6897–6906

    2020

  34. [42]

    La-net: Landmark-aware learning for reliable facial expression recognition under label noise,

    Z. Wu and J. Cui, “La-net: Landmark-aware learning for reliable facial expression recognition under label noise,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 20 698–20 707

    2023

  35. [43]

    Uncertainty-aware label distribution learning for facial expression recognition,

    N. Le, K. Nguyen, Q. Tran, E. Tjiputra, B. Le, and A. Nguyen, “Uncertainty-aware label distribution learning for facial expression recognition,” in Proceedings of the IEEE/CVF winter conference on applications of computer vision , 2023, pp. 6088–6097

    2023

  36. [44]

    Intensity-aware loss for dynamic facial expression recognition in the wild,

    H. Li, H. Niu, Z. Zhu, and F. Zhao, “Intensity-aware loss for dynamic facial expression recognition in the wild,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 37, no. 1, 2023, pp. 67–75

    2023

  37. [45]

    Facial expression recognition boosted by soft label with a diverse ensemble,

    Y . Gan, J. Chen, and L. Xu, “Facial expression recognition boosted by soft label with a diverse ensemble,” Pattern Recognition Letters , vol. 125, pp. 105–112, 2019

    2019

  38. [46]

    A survey of mix-based data augmentation: Taxonomy, methods, applications, and explainability,

    C. Cao, F. Zhou, Y . Dai, J. Wang, and K. Zhang, “A survey of mix-based data augmentation: Taxonomy, methods, applications, and explainability,” ACM Computing Surveys, vol. 57, no. 2, pp. 1–38, 2024. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 13

    2024

  39. [47]

    mixup: Beyond empirical risk minimization,

    H. Zhang, M. Cisse, Y . N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,” inProceedings of the International Conference on Learning Representations (ICLR) , 2018

    2018

  40. [48]

    Vicinal risk minimization,

    O. Chapelle, J. Weston, L. Bottou, and V . Vapnik, “Vicinal risk minimization,” Proceedings of the Advances in Neural Information Processing Systems (NIPS) , vol. 13, 2000

    2000

  41. [49]

    On mixup training: Improved calibration and predictive uncertainty for deep neural networks,

    S. Thulasidasan, G. Chennupati, J. A. Bilmes, T. Bhattacharya, and S. Michalak, “On mixup training: Improved calibration and predictive uncertainty for deep neural networks,” Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) , vol. 32, 2019

    2019

  42. [50]

    AugMix: A simple method to improve robustness and uncertainty under data shift,

    D. Hendrycks*, N. Mu*, E. D. Cubuk, B. Zoph, J. Gilmer, and B. Lak- shminarayanan, “AugMix: A simple method to improve robustness and uncertainty under data shift,” in Proceedings of the International Conference on Learning Representations (ICLR) , 2020

    2020

  43. [51]

    Augmax: Adversarial composition of random augmentations for robust training,

    H. Wang, C. Xiao, J. Kossaifi, Z. Yu, A. Anandkumar, and Z. Wang, “Augmax: Adversarial composition of random augmentations for robust training,” Advances in neural information processing systems , vol. 34, pp. 237–250, 2021

    2021

  44. [52]

    Pixmix: Dreamlike pictures comprehensively improve safety measures,

    D. Hendrycks, A. Zou, M. Mazeika, L. Tang, B. Li, D. Song, and J. Steinhardt, “Pixmix: Dreamlike pictures comprehensively improve safety measures,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 16 783–16 792

    2022

  45. [53]

    Ipmix: Label-preserving data augmentation method for training robust classifiers,

    Z. Huang, X. Bao, N. Zhang, Q. Zhang, X. Tu, B. Wu, and X. Yang, “Ipmix: Label-preserving data augmentation method for training robust classifiers,” Advances in Neural Information Processing Systems, vol. 36, pp. 63 660–63 673, 2023

    2023

  46. [54]

    Diffusemix: Label-preserving data augmentation with diffusion models,

    K. Islam, M. Z. Zaheer, A. Mahmood, and K. Nandakumar, “Diffusemix: Label-preserving data augmentation with diffusion models,” in Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 27 621–27 630

    2024

  47. [55]

    Cutmix: Regularization strategy to train strong classifiers with localizable fea- tures,

    S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, and Y . Yoo, “Cutmix: Regularization strategy to train strong classifiers with localizable fea- tures,” in Proceedings of the International Conference on Computer Vision (ICCV), 2019, pp. 6023–6032

    2019

  48. [56]

    Improved regularization of convolutional neural networks with cutout,

    T. DeVries and G. W. Taylor, “Improved regularization of convolutional neural networks with cutout,” arXiv preprint arXiv:1708.04552 , 2017

    2017 arXiv

  49. [57]

    Smoothmix: a simple yet effective data augmentation to train robust classifiers,

    J.-H. Lee, M. Z. Zaheer, M. Astrid, and S.-I. Lee, “Smoothmix: a simple yet effective data augmentation to train robust classifiers,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops , 2020, pp. 756–757

    2020

  50. [58]

    You only cut once: Boosting data augmentation with a single cut,

    J. Han, P. Fang, W. Li, J. Hong, M. A. Armin, I. Reid, L. Petersson, and H. Li, “You only cut once: Boosting data augmentation with a single cut,” in International Conference on Machine Learning . PMLR, 2022, pp. 8196–8212

    2022

  51. [59]

    Sumix: Mixup with semantic and uncertain information,

    H. Qin, X. Jin, H. Zhu, H. Liao, M. A. El-Yacoubi, and X. Gao, “Sumix: Mixup with semantic and uncertain information,” in European Conference on Computer Vision . Springer, 2024, pp. 70–88

    2024

  52. [60]

    Cutfreq: cut-and-swap frequency components for low-level vision augmentation,

    H. Chen and K. Ma, “Cutfreq: cut-and-swap frequency components for low-level vision augmentation,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 2, 2024, pp. 1072–1080

    2024

  53. [61]

    Gridmix: Strong regularization through local context mapping,

    K. Baek, D. Bang, and H. Shim, “Gridmix: Strong regularization through local context mapping,” Pattern Recognition, vol. 109, p. 107594, 2021

    2021

  54. [62]

    Mixmo: Mixing multiple inputs for multiple outputs via deep subnetworks,

    A. Ram ´e, R. Sun, and M. Cord, “Mixmo: Mixing multiple inputs for multiple outputs via deep subnetworks,” inProceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 823–833

    2021

  55. [63]

    Saliencymix: A saliency guided data augmentation strategy for better regularization,

    A. F. M. S. Uddin, M. S. Monira, W. Shin, T. Chung, and S.-H. Bae, “Saliencymix: A saliency guided data augmentation strategy for better regularization,” in Proceedings of the International Conference on Learning Representations (ICLR) , 2021. [Online]. Available: https://open...

    2021

  56. [64]

    Puzzle mix: Exploiting saliency and local statistics for optimal mixup,

    J.-H. Kim, W. Choo, and H. O. Song, “Puzzle mix: Exploiting saliency and local statistics for optimal mixup,” in Proceedings of the Interna- tional Conference on Machine Learning (ICML) . PMLR, 2020, pp. 5275–5285

    2020

  57. [65]

    Resizemix: Mixing data with preserved object information and true labels,

    J. Qin, J. Fang, Q. Zhang, W. Liu, X. Wang, and X. Wang, “Resizemix: Mixing data with preserved object information and true labels,” arXiv preprint arXiv:2012.11101, 2020

    2012 arXiv

  58. [66]

    Co-mixup: Saliency guided joint mixup with supermodular diversity,

    J.-H. Kim, W. Choo, H. Jeong, and H. O. Song, “Co-mixup: Saliency guided joint mixup with supermodular diversity,” arXiv preprint arXiv:2102.03065, 2021

    2021 arXiv

  59. [67]

    Attentive cutmix: An enhanced data augmentation approach for deep learning based image classification,

    D. Walawalkar, Z. Shen, Z. Liu, and M. Savvides, “Attentive cutmix: An enhanced data augmentation approach for deep learning based image classification,” in Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2020, pp. 3642–3646

    2020

  60. [68]

    Transmix: Attend to mix for vision transformers,

    J.-N. Chen, S. Sun, J. He, P. H. Torr, A. Yuille, and S. Bai, “Transmix: Attend to mix for vision transformers,” in Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 12 135– 12 144

    2022

  61. [69]

    Tokenmix: Rethinking image mixing for data augmentation in vision transformers,

    J. Liu, B. Liu, H. Zhou, H. Li, and Y . Liu, “Tokenmix: Rethinking image mixing for data augmentation in vision transformers,” inEuropean conference on computer vision . Springer, 2022, pp. 455–471

    2022

  62. [70]

    Guidedmixup: an efficient mixup strategy guided by saliency maps,

    M. Kang and S. Kim, “Guidedmixup: an efficient mixup strategy guided by saliency maps,” in Proceedings of the AAAI conference on artificial intelligence, vol. 37, no. 1, 2023, pp. 1096–1104

    2023

  63. [71]

    Manifold mixup: Better representations by interpolating hidden states,

    V . Verma, A. Lamb, C. Beckham, A. Najafi, I. Mitliagkas, D. Lopez-Paz, and Y . Bengio, “Manifold mixup: Better representations by interpolating hidden states,” in Proceedings of the International Conference on Machine Learning (ICML) . PMLR, 2019, pp. 6438–6447

    2019

  64. [72]

    Patchup: A regularization technique for convolutional neural networks,

    M. Faramarzi, M. Amini, A. Badrinaaraayanan, V . Verma, and S. Chan- dar, “Patchup: A regularization technique for convolutional neural networks,” in Proceedings of the Conference on Artificial Intelligence (AAAI), vol. 36, no. 1, 2022, pp. 589–597

    2022

  65. [73]

    Catch-up mix: Catch-up class for struggling filters in cnn,

    M. Kang, M. Kang, and S. Kim, “Catch-up mix: Catch-up class for struggling filters in cnn,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 3, 2024, pp. 2705–2713

    2024

  66. [74]

    Mixgen: A new multi-modal data augmentation,

    X. Hao, Y . Zhu, S. Appalaraju, A. Zhang, W. Zhang, B. Li, and M. Li, “Mixgen: A new multi-modal data augmentation,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2023, pp. 379–389

    2023

  67. [75]

    Contrast and mix: Temporal contrastive video domain adaptation with background mixing,

    A. Sahoo, R. Shah, R. Panda, K. Saenko, and A. Das, “Contrast and mix: Temporal contrastive video domain adaptation with background mixing,” Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), vol. 34, pp. 23 386–23 400, 2021

    2021

  68. [76]

    A case for soft loss functions,

    A. Uma, T. Fornaciari, D. Hovy, S. Paun, B. Plank, and M. Poesio, “A case for soft loss functions,” in Proceedings of the Conference on Human Computation and Crowdsourcing , vol. 8, 2020, pp. 173–177

    2020

  69. [77]

    V oxceleb2: Deep speaker recognition,

    J. S. Chung, A. Nagrani, and A. Zisserman, “V oxceleb2: Deep speaker recognition,” arXiv preprint arXiv:1806.05622 , 2018

    2018 arXiv

  70. [78]

    TSM: Temporal shift module for efficient video understanding,

    J. Lin, C. Gan, and S. Han, “TSM: Temporal shift module for efficient video understanding,” in Proceedings of the International Conference on Computer Vision (CVPR) , 2019, pp. 7083–7093

    2019

  71. [79]

    Imagenet large scale visual recognition challenge,

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein et al., “Imagenet large scale visual recognition challenge,” International Journal of Computer Vision, vol. 115, no. 3, pp. 211–252, 2015

    2015

  72. [80]

    Videomae: Masked autoen- coders are data-efficient learners for self-supervised video pre-training,

    Z. Tong, Y . Song, J. Wang, and L. Wang, “Videomae: Masked autoen- coders are data-efficient learners for self-supervised video pre-training,” Advances in neural information processing systems, vol. 35, pp. 10 078– 10 093, 2022

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.