Pith. sign in

REVIEW 3 major objections 4 minor 65 references

MER-CLIP: AU-Guided Vision-Language Alignment for Micro-Expression Recognition

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By turning facial action-unit labels into muscle-movement sentences and aligning them with video features through CLIP contrastive learning, MER-CLIP claims new state-of-the-art micro-expression recognition on CAS(ME)3 and DFME, with UF1…

desk verdict A well-motivated AU-guided CLIP adaptation for micro-expression recognition with credible mechanism ablations, but the reported SOTA margins rest on test-set-selected hyperparameters and single runs. read the letter →

arxiv 2505.05937 v1 pith:32DCCRTP submitted 2025-05-09 cs.HC

Shifeng Liu , Xinglong Mao , Sirui Zhao , Peiming Li , Tong Xu , Enhong Chen This is my paper · ORCID
classification cs.HC
keywords micro-expressionrecognitionactionunitsFACSvision-languagealignmentCLIPcontrastivelearningdataaugmentationemotion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that micro-expression recognition improves substantially when the subtle facial movements in a video are aligned, through CLIP-style contrastive learning, with text descriptions of the underlying facial action units (AUs) rather than with emotion names. It introduces MER-CLIP, which turns AU labels into action-oriented sentences about muscle movements, feeds them through a frozen CLIP text encoder, and aligns them with spatiotemporal features from a UniformerV2 video encoder. An Emotion Inference Module then maps the aligned motion features to emotion classes, and a purpose-built augmentation, LocalStaticFaceMix, blends each frame with the neutral onset frame of another clip to add facial diversity without destroying the delicate motion. On CAS(ME)3 the model reports UF1 scores of 0.7832, 0.6544, and 0.4997 for 3-, 4-, and 7-class tasks, and on DFME Test A and Test B it reports 0.5024 and 0.5128. If these numbers hold, AU-guided language alignment becomes a new practical route for a task long limited by tiny datasets and hard-to-see motion.

What carries the argument

The load-bearing mechanism is the AU-Guided Cross-Modal Alignment Module: a prompt bank built from action-oriented AU descriptions, a frozen CLIP text encoder, a visual projector, and the CLIP contrastive loss $\mathcal{L}_{\mathrm{CLIP}}$ operating on video-text pairs. What it does is force the video encoder to represent a micro-expression by the muscle movements named in its AU annotation, so fine-grained dynamics that a linear classifier would ignore become the organizing signal; the fixed order of concatenated AU descriptions and the action-oriented wording are both shown to matter for this alignment. Around it sit the Emotion Inference Module, a two-block transformer head that refines the aligned features into emotion semantics, and LocalStaticFaceMix, which blends a selected facial region of every frame with the neutral onset frame of another clip so that identity variation increases without smearing the expression motion. A progressive training schedule starts with only the CLIP loss and gradually shifts weight to the focal classification loss.

What would settle it

Train the same architecture with the AU descriptions replaced by non-semantic but equally structured text, for example the same verbs and nouns randomly reassigned across AUs, while keeping everything else fixed; if UF1 on CAS(ME)3 stays close to 0.7832, the claim that AU semantics drive the alignment is falsified.

Watch

Extended reading notes

Core claim

The central claim is that AU labels are more useful as semantic text than as geometric regions. By rewriting each AU as an action-oriented phrase (for example, "raising the cheeks"), concatenating the active AUs in a fixed order, and slotting the result into templates like "This micro-expression involves {}", the paper obtains text embeddings that a frozen CLIP text encoder can compare with projected video features. The CLIP contrastive loss pulls the visual representation of each clip toward its own AU description and away from others, teaching the video encoder where and how the face moves. The Emotion Inference Module, two transformer blocks plus a linear head, then converts the motion-aligned features into emotion scores, trained with focal loss to handle class imbalance. The authors report that each component contributes: AU alignment is the main driver, replacing it with emotion-name prompts drops performance sharply, action-oriented wording beats FACS terminology, and fixed AU order beats shuffled order.

Load-bearing premise

The whole gain rests on the assumption that the manually written action-oriented AU sentences and their fixed concatenation order give text embeddings whose semantics genuinely match the visual muscle-motion features; if the text branch is only acting as a regularizer, the reported improvements would not transfer to new AU descriptions or datasets.

Editorial extensions

If this is right

  • AU annotations, already present in most micro-expression datasets, become a supervision signal for motion semantics rather than a preprocessing cue, allowing datasets with AU labels to train stronger recognizers without new annotation effort.
  • On CAS(ME)3 the reported 3-class UF1 rises from the best prior 3-class result listed in the paper to 0.7832, and on DFME Test A it rises to 0.5024; if replicated, these are the best results on those splits among the methods compared.
  • Combining LocalStaticFaceMix with AugMix yields consistent gains over either alone, indicating that identity-diversity augmentation that preserves motion is a transferable recipe for small video datasets.
  • The ablation that swaps AU prompts for emotion prompts shows a sharp drop, so the benefit is tied to anatomical motion text, not to the mere presence of a text branch.
  • The progressive training schedule, which starts from pure contrastive alignment and shifts toward classification, suggests that motion-aligned features are a stable initialization for emotion learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the AU-text alignment is what carries the gain, then MER-CLIP could in principle run on clips whose AU labels are predicted rather than annotated, making the method applicable to datasets without FACS coding; this extension is not tested in the paper.
  • The fixed-order finding hints that the text encoder is sensitive to the grammatical ordering of muscle movements, so a learnable prompt or a generative text model could replace the hand-written templates and remove the subjectivity the authors acknowledge.
  • Because the strongest gains appear on the larger CAS(ME)3 and DFME collections, the method's advantage may scale with data; on a small dataset such as CASME II in the 5-class task it trails the top prior method, suggesting the text branch does not fully overcome scarcity.
  • A direct testable extension would be to generate AU descriptions automatically from the video itself and use them as self-supervision, which the authors list as future work.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes MER-CLIP, a vision-language model for micro-expression recognition. A frozen CLIP text encoder converts per-sample AU annotations into action-oriented textual descriptions; these are aligned via a contrastive loss with spatiotemporal features from a UniformerV2 video encoder. An Emotion Inference Module (a lightweight transformer head) and a LocalStaticFaceMix augmentation are added to improve emotion semantics and reduce overfitting. Experiments on SAMM, CASME II, CAS(ME)3, and DFME report large gains, e.g., CAS(ME)3 3-class UF1 0.7832 versus 0.5767 for HTNet, and DFME Test A UF1 0.5024 versus 0.4123 for He et al. Ablations isolate the contribution of each component and of prompt design choices.

Significance. If the reported numbers are reproducible, MER-CLIP would be a clear new state of the art on CAS(ME)3 and DFME and would demonstrate a useful way to inject AU semantics into MER through language. The paper is thoroughly evaluated in terms of ablations: Tables VIII-XI support the proposed mechanism, showing that action-oriented, fixed-order AU descriptions outperform FACS-based or shuffled versions, and that the data augmentation matters. The main reservations concern the experimental protocol rather than the method's internal consistency.

major comments (3)
  1. [Section IV-C and Tables VIII-XI] Section IV-C describes no held-out validation split, and the ablation tables are used to select the number of templates, the augmentation combination, and the final module stack by comparing test-set numbers on DFME Test A/B. Reporting the maximum over configurations evaluated on the test set can inflate expected performance, so the SOTA margins in Tables VI-VII are not yet established. Please rerun with a validation split for model selection and report mean and standard deviation over multiple seeds or over LOSO folds.
  2. [Section III-C and Section IV-C] Equations (5)-(7) construct the text prompt from the AU annotations of each video sequence, and Section IV-C states that the input comprises pairs of trimmed ME video sequences and corresponding textual AU prompts. This means the reported test-set numbers require ground-truth AU labels of the test samples at inference time. Since the compared baselines do not consume per-sample AU labels at test time, the comparison may be unfair. Please either use an AU predictor at test time, or clearly state the test-time annotation requirement and benchmark under identical conditions for all methods.
  3. [Section IV-D and Table IV] The text states that on SAMM 3-class MER-CLIP achieves 'the highest UF1 score of 0.8424', but Table IV reports UF1=0.8321 for the same setup. This internal inconsistency means the numerical claims need verification. Please correct the inconsistency and double-check all reported numbers against the tables.
minor comments (4)
  1. [Section III-C, Eq. (5)] The ordered concatenation operator is typeset as the letter 'N' in the text; please use a proper operator symbol and define it before its first use.
  2. [Table VIII] The header symbols '%' and '!' are not explained in the table caption or text; adding a legend would make the active modules in each row unambiguous.
  3. [Section IV-D] The phrase 'multiple calssification task' contains a typo and should read 'multiple classification tasks'.
  4. [Table VI] The classification task column entries '3', '4', and '7' should be labeled '3-class', '4-class', and '7-class' for clarity, especially since the row labels are otherwise identical.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central claims are empirical benchmark comparisons against external datasets and baselines, and the AU-guided alignment is auxiliary supervision rather than a fitted or self-referential target.

full rationale

The paper's central claim is an empirical performance comparison (UF1/UAR on CAS(ME)3, DFME, SAMM, CASME II), not a quantity derived from its own inputs by construction. The AU-guided cross-modal alignment builds text prompts from AU annotations (Eqs. 5-7) and aligns them with visual features through a CLIP contrastive loss (Eq. 10); this is auxiliary supervision on AU labels, while the final emotion classification loss (Eq. 13) uses independent ground-truth emotion labels. No target metric is defined in terms of a fitted parameter, and no ablated component's gain is a renamed training objective. LocalStaticFaceMix (Eqs. 1-2) preserves the original sequence label and is evaluated empirically in ablations. The self-citations that appear are for the DFME dataset [9], the DFME challenge protocol and its baselines [49], a frame-sampling preprocessing method [42], and related prior MER work [21, 29, 41]; these are external benchmarks, public challenge settings, or preprocessing utilities, not uniqueness theorems or fitted values used to force the outcome. Concerns about test-set-based configuration selection and missing error bars are evaluation-validity issues, not circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The invented components are software modules and text prompts, which are listed as free parameters and axioms rather than entities.

free parameters (6)
  • CLIP contrastive temperature tau = 0.07
    Set by hand; balances the contrastive loss scale in Eq. 10.
  • Focal loss focusing parameter gamma = 2
    Default focal loss setting used to handle class imbalance in Eq. 13.
  • Loss schedule weights lambda_s and lambda_0 = lambda_s=2.0, lambda_0=1.0
    Hand-chosen fixed points for progressive loss weighting in Eqs. 14 and 15.
  • Number of text templates = 7 (chosen from 1, 3, 5, 7, 10)
    Selected by test-set performance in Table XI; no held-out validation described.
  • LocalStaticFaceMix mixing ratio omega = 0.5
    Chosen by hand for blending onset frame region in Eq. 2.
  • Backbone initialization = UniformerV2 pretrained on SSV2
    External pretrained weights; a critical transfer-learning choice for the reported results.
assumptions (5)
  • domain assumption Ground-truth AU annotations are available for all training samples of SAMM, CASME II, CAS(ME)3, and DFME.
    Text prompts in Eqs. 5 and 6 are built from annotated AU sets; Section III-A and Table III confirm datasets provide AU labels.
  • domain assumption The frozen CLIP text encoder produces semantically meaningful embeddings for concatenated action-oriented AU descriptions.
    Cross-modal alignment in Eq. 10 assumes the frozen text encoder maps these short prompts into a space where visual features can be aligned; Section III-C.
  • ad hoc to paper A fixed, ordered concatenation of AU descriptions preserves the semantic structure of the micro-expression.
    The paper chooses ordered concatenation in Eq. 5 and shows in Table X that shuffled order hurts performance.
  • domain assumption The 16-frame temporally adaptive sampling retains onset, apex, and offset frames.
    Section IV-C.2 cites the sampling strategy from ME-PLAN [42] but does not verify it independently for all four datasets.
  • domain assumption The reported benchmark results are representative of model performance under the standard LOSO protocol.
    Single-run results without error bars are treated as stable estimates; Section IV-B defines the protocol.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MER-CLIP: AU-Guided Vision-Language Alignment for Micro-Expression Recognition." pith.science (2026). https://pith.science/paper/32DCCRTP

@misc{pith2026250505937,
  author       = {Pith},
  title        = {Pith review of: MER-CLIP: AU-Guided Vision-Language Alignment for Micro-Expression Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/32DCCRTP}},
  note         = {Machine review of arXiv:2505.05937}
}
read the original abstract

As a critical psychological stress response, micro-expressions (MEs) are fleeting and subtle facial movements revealing genuine emotions. Automatic ME recognition (MER) holds valuable applications in fields such as criminal investigation and psychological diagnosis. The Facial Action Coding System (FACS) encodes expressions by identifying activations of specific facial action units (AUs), serving as a key reference for ME analysis. However, current MER methods typically limit AU utilization to defining regions of interest (ROIs) or relying on specific prior knowledge, often resulting in limited performance and poor generalization. To address this, we integrate the CLIP model's powerful cross-modal semantic alignment capability into MER and propose a novel approach namely MER-CLIP. Specifically, we convert AU labels into detailed textual descriptions of facial muscle movements, guiding fine-grained spatiotemporal ME learning by aligning visual dynamics and textual AU-based representations. Additionally, we introduce an Emotion Inference Module to capture the nuanced relationships between ME patterns and emotions with higher-level semantic understanding. To mitigate overfitting caused by the scarcity of ME data, we put forward LocalStaticFaceMix, an effective data augmentation strategy blending facial images to enhance facial diversity while preserving critical ME features. Finally, comprehensive experiments on four benchmark ME datasets confirm the superiority of MER-CLIP. Notably, UF1 scores on CAS(ME)3 reach 0.7832, 0.6544, and 0.4997 for 3-, 4-, and 7-class classification tasks, significantly outperforming previous methods.

Figures

Figures reproduced from arXiv: 2505.05937 by the authors.

Figure 1
Figure 1. Examples of a MaE (a) and a ME (b) from the same person in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. We illustrate the detailed architecture of MER-CLIP. Before being fed into the video encoder, the input video undergoes a series of augmentations, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The detailed illustration of proposed LocalStaticFaceMix. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The detailed flowchart of the Video Motion Encoding Process. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Confusion matrices of MER tasks on CASME II, SAMM, CAS(ME [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 7
Figure 7. Figure 7: The t-SNE visualizations of ME features for classifying TestA [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 6
Figure 6. Figure 6: GradCam visualizations of several ME samples selected from DFME [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 49 canonical work pages

  1. [1]

    Nonverbal leakage and clues to deception,

    Paul Ekman and Wallace V Friesen, “Nonverbal leakage and clues to deception,” Psychiatry, vol. 32, no. 1, pp. 88–106, 1969

    work page 1969

  2. [2]

    How fast are the leaked facial expressions: The duration of micro- expressions,

    Wen-Jing Yan, Qi Wu, Jing Liang, Yu-Hsin Chen, and Xiaolan Fu, “How fast are the leaked facial expressions: The duration of micro- expressions,” Journal of Nonverbal Behavior , vol. 37, pp. 217–230, 2013

    2013

  3. [3]

    The neuropsychology of facial expression: a review of the neurological and psychological mechanisms for producing facial expressions.,

    William E Rinn, “The neuropsychology of facial expression: a review of the neurological and psychological mechanisms for producing facial expressions.,” Psychological bulletin, vol. 95, no. 1, pp. 52, 1984. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 14

    work page 1984

  4. [4]

    Video-based facial micro-expression analysis: A survey of datasets, features and algorithms,

    Xianye Ben, Yi Ren, Junping Zhang, Su-Jing Wang, Kidiyo Kpalma, Weixiao Meng, and Yong-Jin Liu, “Video-based facial micro-expression analysis: A survey of datasets, features and algorithms,” IEEE transac- tions on pattern analysis and machine intelligence , vol. 44, no. 9, pp. 5826–5846, 2021

    2021

  5. [5]

    Paul Ekman, Telling lies: Clues to deceit in the marketplace, politics, and marriage (revised edition) , WW Norton & Company, 2009

    2009

  6. [6]

    Airport security: Intent to deceive?,

    Sharon Weinberger, “Airport security: Intent to deceive?,” Nature, vol. 465, no. 7297, pp. 412–416, 2010

    work page 2010

  7. [7]

    Emotional expression processing and depressive symptomatology: Eye-tracking reveals differential importance of lower and middle facial areas of interest,

    Laurie Hunter, Laralin Roland, and Ayesha Ferozpuri, “Emotional expression processing and depressive symptomatology: Eye-tracking reveals differential importance of lower and middle facial areas of interest,” Depression research and treatment , vol. 2020, no. 1, pp. 1049851, 2020

    work page 2020

  8. [8]

    Negotiating face-to-face: Men’s facial structure predicts negotiation performance,

    Michael P Haselhuhn, Elaine M Wong, Margaret E Ormiston, M Ena Inesi, and Adam D Galinsky, “Negotiating face-to-face: Men’s facial structure predicts negotiation performance,” The Leadership Quarterly , vol. 25, no. 5, pp. 835–845, 2014

    work page 2014

Show all 65 references
  1. [9]

    Dfme: A new benchmark for dynamic facial micro-expression recognition,

    Sirui Zhao, Huaying Tang, Xinglong Mao, Shifeng Liu, Yiming Zhang, Hao Wang, Tong Xu, and Enhong Chen, “Dfme: A new benchmark for dynamic facial micro-expression recognition,” IEEE Transactions on Affective Computing , 2023

    2023

  2. [10]

    Facial action coding system,

    Paul Ekman and Wallace V Friesen, “Facial action coding system,” Environmental Psychology & Nonverbal Behavior , 1978

    1978

  3. [11]

    Samm: A spontaneous micro-facial movement dataset,

    Adrian K. Davison, Cliff Lansley, Nicholas Costen, Kevin Tan, and Moi Hoon Yap, “Samm: A spontaneous micro-facial movement dataset,” IEEE Transactions on Affective Computing , vol. 9, pp. 116–129, 2018

    2018

  4. [12]

    Casme ii: An improved spontaneous micro-expression database and the baseline evaluation,

    Wen-Jing Yan, Xiaobai Li, Su-Jing Wang, Guoying Zhao, Yong-Jin Liu, Yu-Hsin Chen, and Xiaolan Fu, “Casme ii: An improved spontaneous micro-expression database and the baseline evaluation,” PLoS ONE, vol. 9, 2014

    2014

  5. [13]

    4dme: A spontaneous 4d micro-expression dataset with multimodalities,

    Xiaobai Li, Shiyang Cheng, Yante Li, Muzammil Behzad, Jie Shen, Ste- fanos Zafeiriou, Maja Pantic, and Guoying Zhao, “4dme: A spontaneous 4d micro-expression dataset with multimodalities,” IEEE Transactions on Affective Computing , vol. 14, no. 4, pp. 3031–3047, 2022

    2022

  6. [14]

    Cas (me) 3: A third generation facial spontaneous micro-expression database with depth information and high ecological validity,

    Jingting Li, Zizhao Dong, Shaoyuan Lu, Su-Jing Wang, Wen-Jing Yan, Yinhuan Ma, Ye Liu, Changbing Huang, and Xiaolan Fu, “Cas (me) 3: A third generation facial spontaneous micro-expression database with depth information and high ecological validity,” IEEE Transactions on Patte...

    2022

  7. [15]

    Facial micro- expressions recognition using high speed camera and 3d-gradient de- scriptor,

    Senya Polikovsky, Yoshinari Kameda, and Yuichi Ohta, “Facial micro- expressions recognition using high speed camera and 3d-gradient de- scriptor,” 2009

    2009

  8. [16]

    Recognising spontaneous facial micro-expressions,

    Tomas Pfister, Xiaobai Li, Guoying Zhao, and Matti Pietik ¨ainen, “Recognising spontaneous facial micro-expressions,” in 2011 interna- tional conference on computer vision . IEEE, 2011, pp. 1449–1456

    2011

  9. [17]

    A main directional mean optical flow feature for spontaneous micro-expression recognition,

    Yong-Jin Liu, Jin-Kai Zhang, Wen-Jing Yan, Su-Jing Wang, Guoying Zhao, and Xiaolan Fu, “A main directional mean optical flow feature for spontaneous micro-expression recognition,” IEEE Transactions on Affective Computing, vol. 7, no. 4, pp. 299–310, 2015

    2015

  10. [18]

    Sparse mdmo: Learning a discriminative feature for micro-expression recognition,

    Yong-Jin Liu, Bing-Jun Li, and Yu-Kun Lai, “Sparse mdmo: Learning a discriminative feature for micro-expression recognition,” IEEE Trans- actions on Affective Computing , vol. 12, no. 1, pp. 254–261, 2018

    2018

  11. [19]

    Key facial com- ponents guided micro-expression recognition based on first & second- order motion,

    Yuting Su, Jiaqi Zhang, Jing Liu, and Guangtao Zhai, “Key facial com- ponents guided micro-expression recognition based on first & second- order motion,” in 2021 IEEE International Conference on Multimedia and Expo (ICME) . IEEE, 2021, pp. 1–6

    2021

  12. [20]

    Jgulf: Joint global and unilateral local feature network for micro-expression recognition,

    Fengping Wang, Jie Li, Chun Qi, Lin Wang, and Pan Wang, “Jgulf: Joint global and unilateral local feature network for micro-expression recognition,” Image and Vision Computing, vol. 147, pp. 105091, 2024

    2024

  13. [21]

    A two-stage 3d cnn based learning method for spontaneous micro-expression recognition,

    Sirui Zhao, Hanqing Tao, Yangsong Zhang, Tong Xu, Kun Zhang, Zhong Hao, and Enhong Chen, “A two-stage 3d cnn based learning method for spontaneous micro-expression recognition,” Neurocomputing, vol. 448, pp. 276–289, 2021

    2021

  14. [22]

    Micro-expression recognition from local facial regions,

    Mouath Aouayeb, Wassim Hamidouche, Catherine Soladie, Kidiyo Kpalma, and Renaud Seguier, “Micro-expression recognition from local facial regions,” Signal Processing: Image Communication , vol. 99, pp. 116457, 2021

    2021

  15. [23]

    Deep3dcann: A deep 3dcnn-ann framework for spontaneous micro- expression recognition,

    Selvarajah Thuseethan, Sutharshan Rajasegarar, and John Yearwood, “Deep3dcann: A deep 3dcnn-ann framework for spontaneous micro- expression recognition,” Information Sciences , vol. 630, pp. 341–355, 2023

    2023

  16. [24]

    Joint local and global information learning with single apex frame detection for micro- expression recognition,

    Yante Li, Xiaohua Huang, and Guoying Zhao, “Joint local and global information learning with single apex frame detection for micro- expression recognition,” IEEE Transactions on Image Processing , vol. 30, pp. 249–263, 2020

    2020

  17. [25]

    Feature representation learning with adaptive displacement generation and transformer fusion for micro-expression recognition,

    Zhijun Zhai, Jianhui Zhao, Chengjiang Long, Wenju Xu, Shuangjiang He, and Huijuan Zhao, “Feature representation learning with adaptive displacement generation and transformer fusion for micro-expression recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision...

    2023

  18. [26]

    Mer-gcn: Micro-expression recognition based on relation modeling with graph convolutional networks,

    Ling Lo, Hong-Xia Xie, Hong-Han Shuai, and Wen-Huang Cheng, “Mer-gcn: Micro-expression recognition based on relation modeling with graph convolutional networks,” 2020 IEEE Conference on Mul- timedia Information Processing and Retrieval (MIPR) , pp. 79–84, 2020

    2020

  19. [27]

    Micro-expression recognition based on facial graph representation learning and facial action unit fusion,

    Ling Lei, Tong Chen, Shigang Li, and Jianfeng Li, “Micro-expression recognition based on facial graph representation learning and facial action unit fusion,” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) , pp. 1571–1580, 2021

    2021

  20. [28]

    Micro-expression classifi- cation based on landmark relations with graph attention convolutional network,

    Ankith Jain Rakesh Kumar and Bir Bhanu, “Micro-expression classifi- cation based on landmark relations with graph attention convolutional network,” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1511–1520, 2021

    2021

  21. [29]

    Adaptive graph attention network with temporal fusion for micro-expressions recognition,

    Yiming Zhang, Hao Wang, Yifan Xu, Xinglong Mao, Tong Xu, Sirui Zhao, and Enhong Chen, “Adaptive graph attention network with temporal fusion for micro-expressions recognition,” in 2023 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2023, pp. 1391–1396

    2023

  22. [30]

    Uncovering hidden emo- tions with adaptive multi-attention graph networks,

    Ankith Jain Rakesh Kumar and Bir Bhanu, “Uncovering hidden emo- tions with adaptive multi-attention graph networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 4822–4831

    2024

  23. [31]

    Learning transferable visual models from natural language supervision,

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning....

    2021

  24. [32]

    A brief guide: Code for spontaneous expressions and micro-expressions in videos,

    Zizhao Dong, Gang Wang, Shaoyuan Lu, Wen-Jing Yan, and Su- Jing Wang, “A brief guide: Code for spontaneous expressions and micro-expressions in videos,” in Proceedings of the 1st Workshop on Facial Micro-Expression: Advanced Techniques for Facial Expressions Generation and Spo...

    2021

  25. [33]

    mixup: Beyond empirical risk minimization,

    Hongyi Zhang, “mixup: Beyond empirical risk minimization,” arXiv preprint arXiv:1710.09412, 2017

    2017 arXiv

  26. [34]

    Improved regularization of convolutional neural networks with cutout,

    Terrance DeVries, “Improved regularization of convolutional neural networks with cutout,” arXiv preprint arXiv:1708.04552 , 2017

    2017 arXiv

  27. [35]

    Cutmix: Regularization strategy to train strong classifiers with localizable features,

    Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo, “Cutmix: Regularization strategy to train strong classifiers with localizable features,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 6023– 6032

    2019

  28. [36]

    Uniformerv2: Spatiotemporal learning by arming image vits with video uniformer,

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Limin Wang, and Yu Qiao, “Uniformerv2: Spatiotemporal learning by arming image vits with video uniformer,” arXiv preprint arXiv:2211.09552 , 2022

    2022 arXiv

  29. [37]

    Temporal augmented contrastive learning for micro-expression recognition,

    Tianhao Wang and Lin Shang, “Temporal augmented contrastive learning for micro-expression recognition,” Pattern Recognition Letters, vol. 167, pp. 122–131, 2023

    2023

  30. [38]

    Cmnet: contrastive magnification network for micro- expression recognition,

    Mengting Wei, Xingxun Jiang, Wenming Zheng, Yuan Zong, Cheng Lu, and Jiateng Liu, “Cmnet: contrastive magnification network for micro- expression recognition,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2023, vol. 37, pp. 119–127

    2023

  31. [39]

    Selfme: Self-supervised motion learning for micro-expression recognition,

    Xinqi Fan, Xueli Chen, Mingjie Jiang, Ali Raza Shahid, and Hong Yan, “Selfme: Self-supervised motion learning for micro-expression recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 13834–13843

    2023

  32. [40]

    Micron-bert: Bert-based facial micro-expression recognition,

    Xuan-Bac Nguyen, Chi Nhan Duong, Xin Li, Susan Gauch, Han-Seok Seo, and Khoa Luu, “Micron-bert: Bert-based facial micro-expression recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 1482–1492

    2023

  33. [41]

    Tgmae: Self-supervised micro-expression recognition with temporal gaussian masked autoencoder,

    Shifeng Liu, Xinglong Mao, Sirui Zhao, Chaoyou Fu, Ying Yu, Tong Xu, and Enhong Chen, “Tgmae: Self-supervised micro-expression recognition with temporal gaussian masked autoencoder,” in 2024 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2024, pp. 1–6

    2024

  34. [42]

    Me-plan: A deep prototyp- ical learning with local attention network for dynamic micro-expression recognition,

    Sirui Zhao, Huaying Tang, Shifeng Liu, Yangsong Zhang, Hao Wang, Tong Xu, Enhong Chen, and Cuntai Guan, “Me-plan: A deep prototyp- ical learning with local attention network for dynamic micro-expression recognition,” Neural networks : the official journal of the International ...

    2022

  35. [43]

    Prompting visual-language models for efficient video understanding,

    Chen Ju, Tengda Han, Kunhao Zheng, Ya Zhang, and Weidi Xie, “Prompting visual-language models for efficient video understanding,” in European Conference on Computer Vision . Springer, 2022, pp. 105– 124. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 15

    2022

  36. [44]

    Zero-shot temporal action detection via vision-language prompting,

    Sauradip Nag, Xiatian Zhu, Yi-Zhe Song, and Tao Xiang, “Zero-shot temporal action detection via vision-language prompting,” in European Conference on Computer Vision . Springer, 2022, pp. 681–697

    2022

  37. [45]

    Fine-tuned clip models are efficient video learners,

    Hanoona Rasheed, Muhammad Uzair Khattak, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan, “Fine-tuned clip models are efficient video learners,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 6545–6554

    2023

  38. [46]

    Prompting visual-language models for dynamic facial expression recognition,

    Zengqun Zhao and Ioannis Patras, “Prompting visual-language models for dynamic facial expression recognition,” arXiv preprint arXiv:2308.13382, 2023

    2023 arXiv

  39. [47]

    Augmix: A simple data pro- cessing method to improve robustness and uncertainty,

    Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan, “Augmix: A simple data pro- cessing method to improve robustness and uncertainty,” arXiv preprint arXiv:1912.02781, 2019

    1912 arXiv

  40. [48]

    Focal loss for dense object detection,

    T Lin, “Focal loss for dense object detection,” arXiv preprint arXiv:1708.02002, 2017

    2017 arXiv

  41. [49]

    Dy- namic micro-expression automatic recognition challenge on the fourth chinese conference on affective computing,

    Sirui Zhao, Huaying Tang, Xinglong Mao, and Shifeng Liu, “Dy- namic micro-expression automatic recognition challenge on the fourth chinese conference on affective computing,” https://mea-lab-421.github. io/CCAC-page/, July 10, 2024

    2024

  42. [50]

    The” something something

    Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michalski, Joanna Materzynska, Susanne Westphal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitag, et al., “The” something something” video database for learning and evaluating visual common sense,” in...

    2017

  43. [51]

    Off-apexnet on micro-expression recognition system,

    Yee Siang Gan, Sze-Teng Liong, Wei-Chuen Yau, Yen-Chang Huang, and Lit-Ken Tan, “Off-apexnet on micro-expression recognition system,” Signal Processing: Image Communication , vol. 74, pp. 129–139, 2019

    2019

  44. [52]

    Shal- low triple stream three-dimensional cnn (ststnet) for micro-expression recognition,

    Sze-Teng Liong, Yee Siang Gan, John See, and Huai-Qian Khor, “Shal- low triple stream three-dimensional cnn (ststnet) for micro-expression recognition,” 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019) , pp. 1–5, 2019

    2019

  45. [53]

    Htnet for micro-expression recognition,

    Zhifeng Wang, Kaihao Zhang, Wenhan Luo, and Ramesh Sankara- narayana, “Htnet for micro-expression recognition,” Neurocomputing, vol. 602, pp. 128196, 2024

    2024

  46. [54]

    A novel graph-tcn with a graph structured representation for micro-expression recognition,

    Ling Lei, Jianfeng Li, Tong Chen, and Shigang Li, “A novel graph-tcn with a graph structured representation for micro-expression recognition,” in Proceedings of the 28th ACM International Conference on Multime- dia, 2020, pp. 2237–2245

    2020

  47. [55]

    Merastc: Micro-expression recognition using effective feature encodings and 2d convolutional neural network,

    Puneet Gupta, “Merastc: Micro-expression recognition using effective feature encodings and 2d convolutional neural network,” IEEE Trans- actions on Affective Computing , vol. 14, no. 2, pp. 1431–1441, 2021

    2021

  48. [56]

    Micro-expression recognition with supervised contrastive learning,

    Ruicong Zhi, Jing Hu, and Fei Wan, “Micro-expression recognition with supervised contrastive learning,” Pattern Recognition Letters, vol. 163, pp. 25–31, 2022

    2022

  49. [57]

    C3dbed: Facial micro- expression recognition with three-dimensional convolutional neural net- work embedding in transformer model,

    Hang Pan, Lun Xie, and Zhiliang Wang, “C3dbed: Facial micro- expression recognition with three-dimensional convolutional neural net- work embedding in transformer model,” Engineering Applications of Artificial Intelligence, vol. 123, pp. 106258, 2023

    2023

  50. [58]

    Dlib-ml: A machine learning toolkit,

    Davis E King, “Dlib-ml: A machine learning toolkit,” The Journal of Machine Learning Research , vol. 10, pp. 1755–1758, 2009

    2009

  51. [59]

    Recognizing spontaneous micro-expression using a three-stream convolutional neural network,

    Baolin Song, Ke Li, Yuan Zong, Jie Zhu, Wenming Zheng, Jingang Shi, and Li Zhao, “Recognizing spontaneous micro-expression using a three-stream convolutional neural network,” Ieee Access , vol. 7, pp. 184537–184551, 2019

    2019

  52. [60]

    Revealing the invisible with model and data shrinking for composite-database micro-expression recognition,

    Zhaoqiang Xia, Wei Peng, Huai-Qian Khor, Xiaoyi Feng, and Guoying Zhao, “Revealing the invisible with model and data shrinking for composite-database micro-expression recognition,” IEEE Transactions on Image Processing , vol. 29, pp. 8590–8605, 2020

    2020

  53. [61]

    Feature refinement: An expression-specific feature learning and fusion method for micro-expression recognition,

    Ling Zhou, Qirong Mao, Xiaohua Huang, Feifei Zhang, and Zhihong Zhang, “Feature refinement: An expression-specific feature learning and fusion method for micro-expression recognition,” Pattern Recognition, vol. 122, pp. 108275, 2022

    2022

  54. [62]

    A review of micro-expression recog- nition based on deep learning,

    He Zhang and Hanling Zhang, “A review of micro-expression recog- nition based on deep learning,” in 2022 International Joint Conference on Neural Networks (IJCNN) . IEEE, 2022, pp. 01–08

    2022

  55. [63]

    Sfamnet: A scene flow attention-based micro-expression network,

    Gen-Bing Liong, Sze-Teng Liong, Chee Seng Chan, and John See, “Sfamnet: A scene flow attention-based micro-expression network,” Neurocomputing, vol. 566, pp. 126998, 2024

    2024

  56. [64]

    Grad-cam: Visual explanations from deep networks via gradient-based localization,

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakr- ishna Vedantam, Devi Parikh, and Dhruv Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in Proceedings of the IEEE international conference on computer vision , 2017, pp. 618–626

    2017

  57. [65]

    Visualizing data using t-sne.,

    Laurens Van der Maaten and Geoffrey Hinton, “Visualizing data using t-sne.,” Journal of machine learning research , vol. 9, no. 11, 2008. Shifeng Liu received the B.S degree in the School of Gifted Young from University of Science and Tech- nology of China (USTC). She is curre...

    2008

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.