RECORD
Vision Transformer with Attentive Pooling for Robust Facial Expression Recognition
Computer Vision and Pattern Recognition · Submitted 11 Dec 2022
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Besides, quantitative results on six in-the-wild datasets outperform other state-of-the-art methods.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Facial Expression Recognition (FER) in the wild is an extremely challenging task. Recently, some Vision Transformers (ViT) have been explored for FER, but most of them perform inferiorly compared to Convolutional Neural Networks (CNN). This is mainly because the new proposed modules are difficult to converge well from scratch due to lacking inductive bias and easy to focus on the occlusion and noisy areas. TransFER, a representative transformer-based method for FER, alleviates this with multi-branch attention dropping but brings excessive computations. On the contrary, we present two attentive pooling (AP) modules to pool noisy features directly. The AP modules include Attentive Patch Pooling (APP) and Attentive Token Pooling (ATP). They aim to guide the model to emphasize the most discriminative features while reducing the impacts of less relevant features. The proposed APP is employed to select the most informative patches on CNN features, and ATP discards unimportant tokens in ViT. Being simple to implement and without learnable parameters, the APP and ATP intuitively reduce the computational cost while boosting the performance by ONLY pursuing the most discriminative features. Qualitative results demonstrate the motivations and effectiveness of our attentive poolings. Besides, quantitative results on six in-the-wild datasets outperform other state-of-the-art methods.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Facial Expression Recognition (FER) in the wild is an extremely challenging task. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| s1 | Recently, some Vision Transformers (ViT) have been explored for FER, but most of them perform inferiorly compared to Convolutional Neural Networks (CNN). | background 0.88 | 0.06 | 0.03 | asserted 0.97 |
| s2 | This is mainly because the new proposed modules are difficult to converge well from scratch due to lacking inductive bias and easy to focus on the occlusion and noisy areas. | background 0.71 | 0.11 | 0.03 | asserted 0.96 |
| s3 | TransFER, a representative transformer-based method for FER, alleviates this with multi-branch attention dropping but brings excessive computations. | background 0.89 | 0.06 | 0.03 | asserted 1.00 |
| s4 | On the contrary, we present two attentive pooling (AP) modules to pool noisy features directly. | contribution 0.95 | 0.90 | 0.12 | asserted 1.00 |
| s5 | The AP modules include Attentive Patch Pooling (APP) and Attentive Token Pooling (ATP). | method 0.68 | 0.80 | 0.04 | asserted 0.99 |
| s6 | They aim to guide the model to emphasize the most discriminative features while reducing the impacts of less relevant features. | method 0.70 | 0.46 | 0.10 | asserted 0.93 |
| s7 | The proposed APP is employed to select the most informative patches on CNN features, and ATP discards unimportant tokens in ViT. | method 0.95 | 0.77 | 0.07 | asserted 1.00 |
| s8 | Being simple to implement and without learnable parameters, the APP and ATP intuitively reduce the computational cost while boosting the performance by ONLY pursuing the most discriminative features. | contribution 0.53 | 0.78 | 0.32 | asserted 0.93 |
| s9 | Qualitative results demonstrate the motivations and effectiveness of our attentive poolings. | result 0.99 | 0.92 | 0.07 | asserted 1.00 |
| s10 | Besides, quantitative results on six in-the-wild datasets outperform other state-of-the-art methods. | result 1.00 | 0.94 | 0.51 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Facial Expression Recognition (FER) in the wild is an extremely challenging task. Recently, some Vision Transformers (ViT) have been explored for FER, but most of them perform inferiorly compared to Convolutional Neural Networks (CNN). This is mainly because the new proposed modules are difficult to converge well from scratch due to lacking inductive bias and easy to focus on the occlusion and noisy areas. TransFER, a representative transformer-based method for FER, alleviates this with multi-branch attention dropping but brings excessive computations. On the contrary, we present two attentive pooling (AP) modules to pool noisy features directly. The AP modules include Attentive Patch Pooling (APP) and Attentive Token Pooling (ATP). They aim to guide the model to emphasize the most discriminative features while reducing the impacts of less relevant features. The proposed APP is employed to select the most informative patches on CNN features, and ATP discards unimportant tokens in ViT. Being simple to implement and without learnable parameters, the APP and ATP intuitively reduce the computational cost while boosting the performance by ONLY pursuing the most discriminative features. Qualitative results demonstrate the motivations and effectiveness of our attentive poolings. Besides, quantitative results on six in-the-wild datasets outperform other state-of-the-art methods.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 17 sentences without a confident label
Uses the method or tool of 2 papers
-
The ViT-small [ref] model with 8 Transformer blocks pre-trained on ImageNet[The pre-trained weight is downloaded from <https://github.com/rwightman/pytorch-image-models/>.] is used as Transformer Encoders.
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale · Implementation Details · 0.96
-
The selected patch tokens and the token are jointly fed into the Transformer blocks [ref] which consist of alternating layers of multi-head self-attention (MSA) and MLP blocks to explore the relationship between different patches in a global scope.
Attention Is All You Need · Overview · 0.82
Uses data or values from 1 paper
-
We evaluate our proposed method on six in-the-wild FER datasets: RAF-DB [ref] , FERPlus [ref] , AffectNet [ref] , SFEW [ref] , ExpW [ref] , and Aff-Wild2 [ref] .
Analysing Affective Behavior in the second ABAW2 Competition · Datasets · 0.93
Compares with 2 papers
-
We compare our proposed APViT with other state-of-the-art methods in the second ABAW2 Competition [ref] , as illustrated in Tab. [ref].
Analysing Affective Behavior in the second ABAW2 Competition · Comparison with the State-of-the-Art Methods · 0.83
-
Although GAP outperforms CLT in CVPT [ref] , it fails to achieve a better performance here.
Conditional Positional Encodings for Vision Transformers · Ablation Studies · 0.85
Cites as background 3 papers
-
Recently, the Vision Transformer (ViT) was proposed for image classification [ref] and achieved promising performance with the non-local self-attention mechanism.
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale · Introduction · 0.93
-
Some researchers adopt the ViT for FER [ref] .
MVT: Mask Vision Transformer for Facial Expression Recognition in the wild · Introduction · 0.84
-
The Transformer architecture [ref] was first proposed for machine translation based on attention mechanisms.
Attention Is All You Need · Vision Transformers on FER · 1.00
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.