Pith. sign in

REVIEW 2 major objections 6 minor 52 references

Multimodal Prompt Alignment for Facial Expression Recognition

T0 review · 2 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a fully frozen CLIP model, trained with only 0.443 MB of learnable prompts, outperforms prior facial expression recognition models on RAF-DB, FERPlus, AffectNet-7, and AffectNet-8.

desk verdict Competent prompt-tuning framework for FER whose ablations are honest, but the headline SOTA claim rests on an uncontrolled backbone swap and does not survive the paper's own ViT-B/16 numbers. read the letter →

arxiv 2506.21017 v1 pith:K3JWTCCG submitted 2025-06-26 cs.CV cs.AI

Fuyan Ma , Yiran He , Bin Sun , Shutao Li This is my paper
classification cs.CVcs.AI
keywords facialexpressionrecognitionpromptlearningCLIPvision-languagemodelsmultimodalalignmentLLM-generatedpromptsprototype-guidedfrozenbackbone
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that facial expression recognition can be done by adapting a frozen CLIP model with prompt learning, rather than fine-tuning its image encoder. It claims that three alignment mechanisms—soft prompts matched to LLM-generated hard text descriptions, visual features anchored to class prototypes from the frozen encoder, and global-local cross-modal alignment focused on top facial regions—together push accuracy past state-of-the-art methods on RAF-DB, FERPlus, AffectNet-7, and AffectNet-8. If true, the result matters because it achieves high accuracy with only 0.443 MB of learnable parameters, suggesting CLIP's pretrained representations can be steered to a fine-grained task without retraining the backbone. The paper also reports ablations showing each component contributes a measurable accuracy gain.

What carries the argument

The load-bearing device is the multimodal prompt alignment loss set. Soft-hard prompt alignment treats LLM-generated hard prompts as class weights and pulls learnable soft prompt tokens and their text-encoder embeddings toward them at token level and prompt level. Prototype-guided visual feature alignment regularizes the prompted visual tokens toward class-mean prototypes of the frozen CLIP encoder, preventing the visual prompts from drifting out of the pretrained space. Cross-modal global-local alignment computes logits as global cosine similarity plus the mean of the top 16 local patch similarities with the text embedding, suppressing background.

What would settle it

Train MPA-FER and the strongest CLIP-based baselines, such as E2NT or CEPrompt, with the same backbone, image size, and training budget, and compare on RAF-DB and AffectNet; if the accuracy gap narrows to near zero or reverses, the conclusion that the alignment components drive the gains is falsified. A second check is to replace the LLM-generated descriptions with randomly permuted descriptions; if accuracy does not drop, the claimed role of external semantic knowledge is not supported.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that the full CLIP model, text encoder and image encoder alike, can remain frozen for facial expression recognition and still outperform methods that fine-tune or use trainable adaptation heads. The route is a set of visual prompts inserted at every frozen encoder layer plus textual soft prompts that are trained, under three losses, to absorb knowledge from hard prompts built from a generic template, the class name, and LLM-generated descriptions of each expression. A prototype term keeps prompted visual features near class-mean features computed from the frozen encoder, and a cross-modal term adds sparse top-k local similarities so classification focuses on expression-relevant face regions. On the reported benchmarks the framework reaches 93.74% on RAF-DB, 91.81% on FERPlus, 68.89% on AffectNet-7, and 63.74% on AffectNet-8.

Load-bearing premise

The reported state-of-the-art comparison assumes prior methods are evaluated under comparable conditions, but the paper does not report the backbones of the baselines in the main comparison tables.

Editorial extensions

If this is right

  • Fully frozen CLIP can be adapted to facial expression recognition without fine-tuning the encoders, so the pretrained representation space is preserved throughout training.
  • LLM-generated class descriptions can be injected into trainable prompts through alignment losses, improving accuracy beyond handcrafted templates.
  • Scaling from ViT-B/16 to ViT-L/14 raises accuracy on all four benchmarks while adding only about 0.225 MB of learnable parameters.
  • The framework's total learnable parameters are small enough (0.218 to 0.443 MB) that training is feasible on a single GPU with modest memory.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the backbone is frozen, the same approach should transfer to few-shot or low-label facial expression recognition settings where fine-tuning a large encoder is impractical; a natural test is to measure performance with 1, 4, or 16 training images per class.
  • The top-k local alignment selects expression-relevant patches, so the model's per-class attention maps could serve as a weakly supervised localization signal for which face regions drive each emotion.
  • The LLM descriptions are generated once and can be swapped without retraining the backbone; replacing them with domain-specific or culturally adapted descriptions is a cheap way to probe how much the external text knowledge controls the outcome.
  • The headline margins over existing CLIP-based FER methods may partly reflect the larger ViT-L/14 backbone, since the comparison tables do not report the baselines' backbones; a matched-backbone evaluation would clarify how much the alignment modules themselves contribute.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes MPA-FER, a prompt-learning framework for facial expression recognition that keeps the CLIP text and image encoders fully frozen. It adds per-layer visual prompts, learns soft text prompts that are regularized by LLM-generated hard prompts through token-level and prompt-level alignment, anchors prompted visual features to CLIP-derived class prototypes, and aligns text with global plus top-k local visual features. Experiments on RAF-DB, FERPlus, AffectNet-7 and AffectNet-8 report accuracies of 93.74%, 91.81%, 68.89% and 63.74% with a ViT-L/14 backbone and only 0.443 MB of learnable parameters, together with ablations showing monotone gains for each added component.

Significance. If the state-of-the-art claim survives controlled comparisons, this is a useful contribution: it demonstrates that a fully frozen CLIP can be adapted to FER with a tiny parameter budget, and the component-wise ablations provide evidence that each module matters. The LLM-prior injection and prototype anchoring are simple, interpretable, and consistent with the stated goal of preserving pretrained generalization. The central weakness is that the headline comparisons are not backbone-controlled and lack variance estimates, so the contribution is currently plausible but not established.

major comments (2)
  1. [§4.2, Tables 5-6 and Table 4] Tables 5 and 6 list 'our MPA-FER' results without stating that the numbers come from the ViT-L/14 variant, while Table 4 shows the ViT-B/16 variant obtains 92.51% on RAF-DB, 91.15% on FERPlus and 62.80% on AffectNet-8. For the baselines reported nearby, E2NT (92.63%/91.18%), CEPrompt (92.43%) and POSTER (63.34%) are at or above those ViT-B/16 numbers, and the baselines' backbones and training protocols are not given. Thus the claimed superiority in §4.2 could be due to the larger backbone rather than to the proposed modules. Please report same-backbone, same-protocol comparisons for every baseline, or clearly separate backbone effects from method effects.
  2. [Tables 1-6] All accuracy tables report single numbers; no standard deviations, seeds, or significance tests are given. On AffectNet-8 the advantage over POSTER is 0.40 percentage points, and on AffectNet-7 the margin over POSTER is 1.58 but with unknown baseline settings; without repeated runs it is not possible to judge whether the margins are meaningful. Please report mean ± standard deviation over at least three seeds for the main comparisons, and indicate whether the reported numbers are best or mean.
minor comments (6)
  1. [Eqs. (1)-(2)] The index d/c is confusing: P(y_d | t_d) is defined with class d, but Eq. (2) sums over c with log P(y_c | t_d). Please clarify whether the soft prompt embedding is shared or class-specific and align the indices.
  2. [§4, Implementation Details] The paper does not state whether the main results use the full training set for prototype computation; Table 3's 'Full' row matches Table 4 numbers, but the choice should be stated explicitly in the implementation details.
  3. [§3.2] The exact LLM-generated descriptions are not provided, which limits reproducibility; consider giving the full prompt template and the generated descriptions in an appendix or supplementary material.
  4. [References / Abstract] Reference [1] is a GPT-4 technical report, but the text cites 'ChatGPT-3.5[1]' in the Introduction; also the abstract says 'three FER benchmark datasets' while four benchmark settings (RAF-DB, FERPlus, AffectNet-7, AffectNet-8) are used.
  5. [Table 2] The caption should define what 'MPA-FER + (1)' includes; without that definition, the rows with and without 'Soft-hard Prompt Align.' are easy to misread.
  6. [Introduction, Contributions] The claim of being 'the first to explore' a fully frozen CLIP for FER should be softened or substantiated with a comparison to prior frozen-backbone VLM works.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation is self-contained and all losses are training objectives with fixed or external anchors.

full rationale

The central derivation chain (Sec. 3) consists of four training losses, none of which is a pre-fitted constant or a renamed version of the reported accuracy. The LLM hard prompts (Sec. 3.2) are external knowledge generated by ChatGPT-3.5, not derived from the FER benchmark results. The soft-hard alignment losses L_ta and L_pa (Eqs. 1-4) are contrastive objectives that regularize soft prompts toward hard prompt embeddings; they do not by construction determine the test accuracy. The prototypes in Eq. (6) are class means of frozen CLIP features from a training subset and are used only as fixed anchors in the regularizer L_v (Eq. 7); this is an anchoring mechanism, not a circular prediction: prompted visual features are trained toward a fixed target derived from a frozen encoder, and classification logits (Eqs. 8-10) still depend on the learned prompts. The cross-modal global-local alignment is a standard top-k similarity pooling loss. The self-citations to Refs. [23] and [24] are prior method baselines, not load-bearing justifications of the core claim. The only substantive concern is the SOTA comparison in Tables 5-6, where MPA-FER uses ViT-L/14 while baseline backbones are not reported; that is an experimental-fairness or correctness risk, not a circularity in the derivation chain. Accordingly, no circular step meeting the quote-and-reduction bar is present.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The framework does not introduce new physical entities. The trainable quantities are prompts and the hyperparameters listed above. The key assumptions are the reliability of frozen CLIP features and of LLM text descriptions as anchors for expression semantics.

free parameters (6)
  • beta (loss weight for textual alignment) = 1
    Chosen experimentally as balance factor in Eq. (12); not justified by a sweep.
  • gamma (loss weight for prototype alignment) = 1
    Chosen experimentally in Eq. (12); no sensitivity analysis shown.
  • k (top-k local features) = 16
    Set for local feature selection in Eq. (9); no ablation on k is reported.
  • N_p (visual prompts per layer) = 8
    Selected as the number of visual prompt tokens; no ablation reported.
  • text prompt length = 10
    Selected as the number of learnable soft prompt tokens; no ablation reported.
  • temperature tau in Eqs. (1) and (3) = not stated
    Used in cosine-similarity softmax; value not disclosed in the paper.
assumptions (3)
  • domain assumption Frozen CLIP features form a semantically meaningful space for facial expression recognition.
    The method keeps both CLIP encoders frozen and anchors prompted features to CLIP prototypes (Section 3.4); if CLIP representations are not discriminative for expressions, the framework cannot succeed.
  • domain assumption LLM-generated descriptions are accurate and class-discriminative for all three datasets.
    The hard prompts used to regularize soft prompts (Section 3.2 and Eqs. (1)-(4)) are generated by ChatGPT-3.5 with a single generic query; no verification or per-dataset curation is described.
  • domain assumption The hard top-k selection in Eq. (9) provides a usable gradient signal for training.
    The indicator function is non-differentiable, but the paper does not describe a gradient approximation or a soft-top-k alternative; the training dynamics rely on the masked similarities being stable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multimodal Prompt Alignment for Facial Expression Recognition." pith.science (2026). https://pith.science/paper/K3JWTCCG

@misc{pith2026250621017,
  author       = {Pith},
  title        = {Pith review of: Multimodal Prompt Alignment for Facial Expression Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K3JWTCCG}},
  note         = {Machine review of arXiv:2506.21017}
}
read the original abstract

Prompt learning has been widely adopted to efficiently adapt vision-language models (VLMs) like CLIP for various downstream tasks. Despite their success, current VLM-based facial expression recognition (FER) methods struggle to capture fine-grained textual-visual relationships, which are essential for distinguishing subtle differences between facial expressions. To address this challenge, we propose a multimodal prompt alignment framework for FER, called MPA-FER, that provides fine-grained semantic guidance to the learning process of prompted visual features, resulting in more precise and interpretable representations. Specifically, we introduce a multi-granularity hard prompt generation strategy that utilizes a large language model (LLM) like ChatGPT to generate detailed descriptions for each facial expression. The LLM-based external knowledge is injected into the soft prompts by minimizing the feature discrepancy between the soft prompts and the hard prompts. To preserve the generalization abilities of the pretrained CLIP model, our approach incorporates prototype-guided visual feature alignment, ensuring that the prompted visual features from the frozen image encoder align closely with class-specific prototypes. Additionally, we propose a cross-modal global-local alignment module that focuses on expression-relevant facial features, further improving the alignment between textual and visual features. Extensive experiments demonstrate our framework outperforms state-of-the-art methods on three FER benchmark datasets, while retaining the benefits of the pretrained model and minimizing computational costs.

Figures

Figures reproduced from arXiv: 2506.21017 by the authors.

Figure 1
Figure 1. (a) Conventional CLIP-based FER paradigm that adapts [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of our proposed multimodal prompt alignment framework, MPA-FER. The multi-granularity hard prompts consist [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The attention visualization of CoOp, MPA-FER without [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: t-SNE visualizations of the learned feature representations of the test set of RAF-DB. Subfigure (a) illustrates the results when [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 40 canonical work pages

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    arXiv

  2. [2]

    Flamingo: a visual language model for few-shot learning.Advances in Neural Information Processing Systems, 35:23716–23736,

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning.Advances in Neural Information Processing Systems, 35:23716–23736,

  3. [3]

    Training deep networks for facial ex- pression recognition with crowd-sourced label distribution

    Emad Barsoum, Cha Zhang, Cristian Canton Ferrer, and Zhengyou Zhang. Training deep networks for facial ex- pression recognition with crowd-sourced label distribution. InProceedings of ACM International Conference on Multi- modal Interaction, pages 279–283, 2016. 6

    work page 2016

  4. [4]

    Xcoop: Explainable prompt learning for computer-aided di- agnosis via concept-guided context optimization

    Yequan Bie, Luyang Luo, Zhixuan Chen, and Hao Chen. Xcoop: Explainable prompt learning for computer-aided di- agnosis via concept-guided context optimization. InIn- ternational Conference on Medical Image Computing and Computer-Assisted Intervention, pages 773–783. Springer,

    work page

  5. [5]

    Impact of deep learning ap- proaches on facial expression recognition in healthcare in- dustries.IEEE Transactions on Industrial Informatics, 2022

    Carmen Bisogni, Aniello Castiglione, Sanoar Hossain, Fabio Narducci, and Saiyed Umer. Impact of deep learning ap- proaches on facial expression recognition in healthcare in- dustries.IEEE Transactions on Industrial Informatics, 2022. doi: 10.1109/TII.2022.3141400. 1

    work page arXiv 2022

  6. [6]

    Lasp: Text-to- text optimization for language-aware soft prompting of vi- sion & language models

    Adrian Bulat and Georgios Tzimiropoulos. Lasp: Text-to- text optimization for language-aware soft prompting of vi- sion & language models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23232–23241, 2023. 3, 4

    work page 2023

  7. [7]

    Histograms of oriented gradi- ents for human detection

    Navneet Dalal and Bill Triggs. Histograms of oriented gradi- ents for human detection. InIEEE Conference on Computer Vision and Pattern Recognition, pages 886–893, 2005. 2

    work page 2005

  8. [8]

    An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. 1

    arXiv 2010

Show all 52 references
  1. [9]

    Scaling up visual and vision-language representa- tion learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representa- tion learning with noisy text supervision. InInternational Conference on Machine Learning, pages 4904–4916. PMLR,

  2. [10]

    Maple: Multi-modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19113–19122, 2023. 2

    2023

  3. [11]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. InProceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 2

    2023

  4. [12]

    Backpropagation applied to handwritten zip code recognition.Neural Computation, 1(4):541–551,

    Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation applied to handwritten zip code recognition.Neural Computation, 1(4):541–551,

  5. [13]

    Mvit: Mask vision transformer for fa- cial expression recognition in the wild.arXiv preprint arXiv:2106.04520, 2021

    Hanting Li, Mingzhe Sui, Feng Zhao, Zhengjun Zha, and Feng Wu. Mvit: Mask vision transformer for fa- cial expression recognition in the wild.arXiv preprint arXiv:2106.04520, 2021. 8

    2021 arXiv

  6. [14]

    Adaptively learning facial expression represen- tation via cf labels and distillation.IEEE Transactions on Image Processing, 30:2016–2028, 2021

    Hangyu Li, Nannan Wang, Xinpeng Ding, Xi Yang, and Xinbo Gao. Adaptively learning facial expression represen- tation via cf labels and distillation.IEEE Transactions on Image Processing, 30:2016–2028, 2021. 1, 3, 8

    2016

  7. [15]

    Cliper: A unified vision-language framework for in-the-wild facial expression recognition

    Hanting Li, Hongjing Niu, Zhaoqing Zhu, and Feng Zhao. Cliper: A unified vision-language framework for in-the-wild facial expression recognition. In2024 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE, 2024. 2, 3, 8

    2024

  8. [16]

    Knowledge-enhanced facial expression recognition with emotional-to-neutral transformation.arXiv preprint arXiv:2409.08598, 2024

    Hangyu Li, Yihan Xu, Jiangchao Yao, Nannan Wang, Xinbo Gao, and Bo Han. Knowledge-enhanced facial expression recognition with emotional-to-neutral transformation.arXiv preprint arXiv:2409.08598, 2024. 2, 3, 8

    2024 arXiv

  9. [17]

    Reliable crowdsourcing and deep locality-preserving learning for unconstrained facial ex- pression recognition.IEEE Transactions on Image Process- ing, 28(1):356–370, 2018

    Shan Li and Weihong Deng. Reliable crowdsourcing and deep locality-preserving learning for unconstrained facial ex- pression recognition.IEEE Transactions on Image Process- ing, 28(1):356–370, 2018. 1, 3, 6

    2018

  10. [18]

    Deep facial expression recogni- tion: A survey.IEEE Transactions on Affective Computing,

    Shan Li and Weihong Deng. Deep facial expression recogni- tion: A survey.IEEE Transactions on Affective Computing,

  11. [19]

    Oc- clusion aware facial expression recognition using cnn with attention mechanism.IEEE Transactions on Image Process- ing, 28(5):2439–2450, 2018

    Yong Li, Jiabei Zeng, Shiguang Shan, and Xilin Chen. Oc- clusion aware facial expression recognition using cnn with attention mechanism.IEEE Transactions on Image Process- ing, 28(5):2439–2450, 2018. 3

    2018

  12. [20]

    Fer-former: Multimodal transformer for facial ex- pression recognition.IEEE Transactions on Multimedia,

    Yande Li, Mingjie Wang, Minglun Gong, Yonggang Lu, and Li Liu. Fer-former: Multimodal transformer for facial ex- pression recognition.IEEE Transactions on Multimedia,

  13. [21]

    Nor- face: Improving facial expression analysis by identity nor- malization

    Hanwei Liu, Rudong An, Zhimeng Zhang, Bowen Ma, Wei Zhang, Yan Song, Yujing Hu, Wei Chen, and Yu Ding. Nor- face: Improving facial expression analysis by identity nor- malization. InEuropean Conference on Computer Vision, pages 293–314. Springer, 2024. 1

    2024

  14. [22]

    The extended cohn- kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression

    Patrick Lucey, Jeffrey F Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews. The extended cohn- kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. InIEEE Conference on Com- puter Vision and Pattern Recognition-workshops, ...

    2010

  15. [23]

    Facial expression recog- nition with visual transformers and attentional selective fu- sion.IEEE Transactions on Affective Computing, 2021

    Fuyan Ma, Bin Sun, and Shutao Li. Facial expression recog- nition with visual transformers and attentional selective fu- sion.IEEE Transactions on Affective Computing, 2021. doi: 10.1109/TAFFC.2021.3122146. 1, 3, 8

    2021

  16. [24]

    Transformer-augmented network with online label correction for facial expression recognition.IEEE Transactions on Affective Computing, 15 (2):593–605, 2023

    Fuyan Ma, Bin Sun, and Shutao Li. Transformer-augmented network with online label correction for facial expression recognition.IEEE Transactions on Affective Computing, 15 (2):593–605, 2023. 3, 8 9

    2023

  17. [25]

    Locoop: Few-shot out-of-distribution detection via prompt learning.Advances in Neural Information Processing Sys- tems, 36:76298–76310, 2023

    Atsuyuki Miyai, Qing Yu, Go Irie, and Kiyoharu Aizawa. Locoop: Few-shot out-of-distribution detection via prompt learning.Advances in Neural Information Processing Sys- tems, 36:76298–76310, 2023. 3

    2023

  18. [26]

    Affectnet: A database for facial expression, valence, and arousal computing in the wild.IEEE Transactions on Affective Computing, 10(1):18–31, 2017

    Ali Mollahosseini, Behzad Hasani, and Mohammad H Ma- hoor. Affectnet: A database for facial expression, valence, and arousal computing in the wild.IEEE Transactions on Affective Computing, 10(1):18–31, 2017. 6

    2017

  19. [27]

    Prompt learning via meta-regularization

    Jinyoung Park, Juyeon Ko, and Hyunwoo J Kim. Prompt learning via meta-regularization. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26940–26950, 2024. 4

    2024

  20. [28]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. InInternational Conference on Machine Learning,...

    2021

  21. [29]

    Ro- bust facial expression recognition using local binary pat- terns

    Caifeng Shan, Shaogang Gong, and Peter W McOwan. Ro- bust facial expression recognition using local binary pat- terns. InIEEE International Conference on Image Process- ing, pages II–370, 2005. 2

    2005

  22. [30]

    Vl-bert: Pre-training of generic visual- linguistic representations.arXiv preprint arXiv:1908.08530,

    Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. Vl-bert: Pre-training of generic visual- linguistic representations.arXiv preprint arXiv:1908.08530,

    1908 arXiv

  23. [31]

    Dualcoop: Fast adaptation to multi-label recognition with limited annota- tions.Advances in Neural Information Processing Systems, 35:30569–30582, 2022

    Ximeng Sun, Ping Hu, and Kate Saenko. Dualcoop: Fast adaptation to multi-label recognition with limited annota- tions.Advances in Neural Information Processing Systems, 35:30569–30582, 2022. 3

    2022

  24. [32]

    Induced disgust, happi- ness and surprise: an addition to the mmi facial expres- sion database

    Michel Valstar and Maja Pantic. Induced disgust, happi- ness and surprise: an addition to the mmi facial expres- sion database. InProceedings of International Workshop on EMOTION: Corpora for Research on Emotion and Affect, pages 65–70, 2010. 1

    2010

  25. [33]

    An investigation of basic facial expression recognition in autism spectrum disorders.Cognition and Emotion, 22(7):1353– 1380, 2008

    Simon Wallace, Michael Coleman, and Anthony Bailey. An investigation of basic facial expression recognition in autism spectrum disorders.Cognition and Emotion, 22(7):1353– 1380, 2008. 1

    2008

  26. [34]

    Suppressing uncertainties for large-scale facial ex- pression recognition

    Kai Wang, Xiaojiang Peng, Jianfei Yang, Shijian Lu, and Yu Qiao. Suppressing uncertainties for large-scale facial ex- pression recognition. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 6897–6906, 2020. 1, 8

    2020

  27. [35]

    Region attention networks for pose and occlusion robust facial expression recognition.IEEE Transactions on Image Processing, 29:4057–4069, 2020

    Kai Wang, Xiaojiang Peng, Jianfei Yang, Debin Meng, and Yu Qiao. Region attention networks for pose and occlusion robust facial expression recognition.IEEE Transactions on Image Processing, 29:4057–4069, 2020. 1, 8

    2020

  28. [36]

    Pose-aware facial expression recognition as- sisted by expression descriptions.IEEE Transactions on Af- fective Computing, 15(1):241–253, 2023

    Shangfei Wang, Yi Wu, Yanan Chang, Guoming Li, and Meng Mao. Pose-aware facial expression recognition as- sisted by expression descriptions.IEEE Transactions on Af- fective Computing, 15(1):241–253, 2023. 2

    2023

  29. [37]

    Vita-clip: Video and text adaptive clip via multimodal prompting

    Syed Talal Wasim, Muzammal Naseer, Salman Khan, Fa- had Shahbaz Khan, and Mubarak Shah. Vita-clip: Video and text adaptive clip via multimodal prompting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 23034–23044, 2023. 2

    2023

  30. [38]

    Facial expression recognition for human- robot interaction–a prototype

    Matthias Wimmer, Bruce A MacDonald, Dinuka Jayamuni, and Arpit Yadav. Facial expression recognition for human- robot interaction–a prototype. InInternational Workshop on Robot Vision, pages 139–152. Springer, 2008. 1

    2008

  31. [39]

    Trans- fer: Learning relation-aware facial expression representa- tions with transformers

    Fanglei Xue, Qiangchang Wang, and Guodong Guo. Trans- fer: Learning relation-aware facial expression representa- tions with transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3601– 3610, 2021. 3, 8

    2021

  32. [40]

    Vision transformer with attentive pooling for robust facial expression recognition

    Fanglei Xue, Qiangchang Wang, Zichang Tan, Zhong- song Ma, and Guodong Guo. Vision transformer with attentive pooling for robust facial expression recognition. IEEE Transactions on Affective Computing, 2022. doi: 10.1109/TAFFC.2022.3226473. 3, 8

    2022

  33. [41]

    Visual- language prompt tuning with knowledge-guided context op- timization

    Hantao Yao, Rui Zhang, and Changsheng Xu. Visual- language prompt tuning with knowledge-guided context op- timization. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6757– 6767, 2023. 3

    2023

  34. [42]

    Facial expres- sion recognition with inconsistently annotated datasets

    Jiabei Zeng, Shiguang Shan, and Xilin Chen. Facial expres- sion recognition with inconsistently annotated datasets. In Proceedings of the European Conference on Computer Vi- sion, pages 222–237, 2018. 1

    2018

  35. [43]

    Weakly- supervised facial expression recognition in the wild with noisy data.IEEE Transactions on Multimedia, 24:1800– 1814, 2021

    Feifei Zhang, Mingliang Xu, and Changsheng Xu. Weakly- supervised facial expression recognition in the wild with noisy data.IEEE Transactions on Multimedia, 24:1800– 1814, 2021. 1

    2021

  36. [44]

    Weakly-supervised text-driven contrastive learning for facial behavior understanding

    Xiang Zhang, Taoyue Wang, Xiaotian Li, Huiyuan Yang, and Lijun Yin. Weakly-supervised text-driven contrastive learning for facial behavior understanding. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion (ICCV), pages 20751–20762, 2023. 3, 8

    2023

  37. [45]

    Facial expression recognition from near- infrared videos.Image and Vision Computing, 29(9):607– 619, 2011

    Guoying Zhao, Xiaohua Huang, Matti Taini, Stan Z Li, and Matti Pietik¨aInen. Facial expression recognition from near- infrared videos.Image and Vision Computing, 29(9):607– 619, 2011. 1

    2011

  38. [46]

    Prompting visual- language models for dynamic facial expression recognition

    Zengqun Zhao and Ioannis Patras. Prompting visual- language models for dynamic facial expression recognition. arXiv preprint arXiv:2308.13382, 2023. 2

    2023 arXiv

  39. [47]

    Learning deep global multi-scale and local attention features for facial expression recognition in the wild.IEEE Transactions on Image Processing, 30:6544–6556, 2021

    Zengqun Zhao, Qingshan Liu, and Shanmin Wang. Learning deep global multi-scale and local attention features for facial expression recognition in the wild.IEEE Transactions on Image Processing, 30:6544–6556, 2021. 3, 8

    2021

  40. [48]

    Poster: A pyra- mid cross-fusion transformer network for facial expression recognition

    Ce Zheng, Matias Mendieta, and Chen Chen. Poster: A pyra- mid cross-fusion transformer network for facial expression recognition. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3146–3155, 2023. 8

    2023

  41. [49]

    Ceprompt: Cross-modal emotion-aware prompt- ing for facial expression recognition.IEEE Transactions on Circuits and Systems for Video Technology, 2024

    Haoliang Zhou, Shucheng Huang, Feifei Zhang, and Chang- sheng Xu. Ceprompt: Cross-modal emotion-aware prompt- ing for facial expression recognition.IEEE Transactions on Circuits and Systems for Video Technology, 2024. 2, 3, 8

    2024

  42. [50]

    Conditional prompt learning for vision-language mod- els

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language mod- els. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 16816–16825,

  43. [51]

    Learning to prompt for vision-language models.In- ternational Journal of Computer Vision, 130(9):2337–2348,

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models.In- ternational Journal of Computer Vision, 130(9):2337–2348,

  44. [2020]

    doi: 10.1109/TAFFC.2020.2981446. 1

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.