RECORD
Facial Expression Recognition and Image Description Generation in Vietnamese
Computer Vision and Pattern Recognition · Submitted 12 Aug 2022
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Finally, we combine the results of all models to create sentences that describe the visual content and the human emotions in the images.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
This paper discusses a facial expression recognition model and a description generation model to build descriptive sentences for images and facial expressions of people in images. Our study shows that YOLOv5 achieves better results than a traditional CNN for all emotions on the KDEF dataset. In particular, the accuracies of the CNN and YOLOv5 models for emotion recognition are 0.853 and 0.938, respectively. A model for generating descriptions for images based on a merged architecture is proposed using VGG16 with the descriptions encoded over an LSTM model. YOLOv5 is also used to recognize dominant colors of objects in the images and correct the color words in the descriptions generated if it is necessary. If the description contains words referring to a person, we recognize the emotion of the person in the image. Finally, we combine the results of all models to create sentences that describe the visual content and the human emotions in the images. Experimental results on the Flickr8k dataset in Vietnamese achieve BLEU-1, BLEU-2, BLEU-3, BLEU-4 scores of 0.628; 0.425; 0.280; and 0.174, respectively.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | This paper discusses a facial expression recognition model and a description generation model to build descriptive sentences for images and facial expressions of people in images. | contribution 0.91 | 0.39 | 0.07 | asserted 0.79 |
| s1 | Our study shows that YOLOv5 achieves better results than a traditional CNN for all emotions on the KDEF dataset. | result 1.00 | 0.95 | 0.29 | asserted 1.00 |
| s2 | In particular, the accuracies of the CNN and YOLOv5 models for emotion recognition are 0.853 and 0.938, respectively. | result 1.00 | 0.92 | 0.14 | asserted 1.00 |
| s3 | A model for generating descriptions for images based on a merged architecture is proposed using VGG16 with the descriptions encoded over an LSTM model. | method 0.90 | 0.80 | 0.05 | asserted 0.99 |
| s4 | YOLOv5 is also used to recognize dominant colors of objects in the images and correct the color words in the descriptions generated if it is necessary. | method 0.99 | 0.82 | 0.03 | asserted 0.99 |
| s5 | If the description contains words referring to a person, we recognize the emotion of the person in the image. | method 1.00 | 0.71 | 0.04 | asserted 0.93 |
| s6 | Finally, we combine the results of all models to create sentences that describe the visual content and the human emotions in the images. | method 0.93 | 0.89 | 0.30 | asserted 0.97 |
| s7 | Experimental results on the Flickr8k dataset in Vietnamese achieve BLEU-1, BLEU-2, BLEU-3, BLEU-4 scores of 0.628; 0.425; 0.280; and 0.174, respectively. | result 1.00 | 0.95 | 0.19 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
This paper discusses a facial expression recognition model and a description generation model to build descriptive sentences for images and facial expressions of people in images. Our study shows that YOLOv5 achieves better results than a traditional CNN for all emotions on the KDEF dataset. In particular, the accuracies of the CNN and YOLOv5 models for emotion recognition are 0.853 and 0.938, respectively. A model for generating descriptions for images based on a merged architecture is proposed using VGG16 with the descriptions encoded over an LSTM model. YOLOv5 is also used to recognize dominant colors of objects in the images and correct the color words in the descriptions generated if it is necessary. If the description contains words referring to a person, we recognize the emotion of the person in the image. Finally, we combine the results of all models to create sentences that describe the visual content and the human emotions in the images. Experimental results on the Flickr8k dataset in Vietnamese achieve BLEU-1, BLEU-2, BLEU-3, BLEU-4 scores of 0.628; 0.425; 0.280; and 0.174, respectively.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 3 sentences without a confident label
Uses the method or tool of 1 paper
-
We use the merge architecture to construct descriptions for images [ref] with the VGG16 model for extracting image features and the LSTM model for constructing image caption, as presented in Figure [ref].
What is the Role of Recurrent Neural Networks (RNNs) in an Image Caption Generator? · Image Description Generation Model · 1.00
Cites as background 1 paper
-
For the last category, there are 2 architectures to generate images captions: inject and merge architectures [ref] .
What is the Role of Recurrent Neural Networks (RNNs) in an Image Caption Generator? · Introduction · 0.83
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.