Context-Aware Emotion Recognition Networks

Jiyoung Lee; Jungin Park; Kwanghoon Sohn; Seungryong Kim; Sunok Kim

REVIEW 4 major objections 5 minor 53 references

Context-Aware Emotion Recognition Networks

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Hiding the face and attending to the scene improves emotion recognition accuracy over face-only models.

desk verdict A useful new context-aware emotion video dataset and a plausible context-boost result, but the paper's main table is compromised by a likely show-level train/test leak and an underspecified face-hiding step. read the letter →

arxiv 1908.05913 v1 pith:2OVYUPSU submitted 2019-08-16 cs.CV cs.HCcs.MM

Jiyoung Lee , Seungryong Kim , Sunok Kim , Jungin Park , Kwanghoon Sohn This is my paper · ORCID

classification cs.CVcs.HCcs.MM

keywords emotionrecognitioncontext-awareattentionmechanismface-hiddentrainingadaptivefusionCAERbenchmarkvideodeepconvolutionalnetworks

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that reading emotion from a face alone misses too much, because ambiguous expressions are resolved by what surrounds them. It proposes CAER-Net, a two-stream network that encodes the cropped face and the face-hidden scene separately, learns where in the scene to look through an attention mechanism, and fuses the two with adaptively learned weights. On the new CAER benchmark the full model reaches 77.04% accuracy versus 74.13% for the face-only stream, and training on CAER plus AFEW lifts AFEW accuracy from 41.86% to 51.68%. The paper also contributes CAER, a dataset of 13,201 TV-show clips annotated with seven emotion categories and confidence scores, built to make context-aware emotion recognition testable at scale.

What carries the argument

The load-bearing mechanism is CAER-Net's two-stream design with its two attention stages. The context stream receives face-hidden inputs $V_C$; an attention inference module converts an intermediate feature map into a spatially normalized attention map via spatial softmax, $\hat{A}_i = \exp(A_i)/\sum_j \exp(A_j)$, and applies it as $\bar{X}_C = \hat{A} \odot X_C$, so the network focuses on salient non-face regions. The adaptive fusion network then computes scalar weights $\lambda_F$ and $\lambda_C$ for the face and context features, concatenates the weighted features, and classifies them. The face-hiding step is what forces the context stream to discover scene signals instead of collapsing onto the face.

What would settle it

Train CAER-Net with a specified face-hiding step (black rectangle, blur, inpainting, or pixelation) and check whether the reported face-only-to-full accuracy gap of about 2.9 points on CAER holds across all variants; if the gap disappears under any transparent hiding method, or if the hidden inputs still contain detectable face pixels, the central claim that scene context drives the gain is not settled.

Watch

Extended reading notes

Core claim

The central discovery is that scene context carries emotional signal that facial expression alone does not, and a network can capture it by deliberately hiding the face and asking an attention module where to look. The architecture has a face stream on cropped faces and a context stream on face-hidden frames; the context features are multiplied by a spatial softmax attention map, and another attention module produces per-sample fusion weights $\lambda_F$ and $\lambda_C$ with $\lambda_F+\lambda_C=1$. The decisive evidence is the ablation: face-only is 74.13%, and adding the context stream reaches 77.04% on CAER, with the context stream alone already at 71.94%. On AFEW, training on CAER+AFEW raises accuracy from 41.86% (face-only) to 51.68%.

Load-bearing premise

The context stream is trained on face-hidden inputs, but the paper never says how the faces are hidden; if that hidden preprocessing leaks face identity or removes needed scene structure, the entire claimed advantage of context attention could be an artifact of that choice.

Editorial extensions

If this is right

Scene context should be treated as a first-class input for emotion recognition in the wild, not as a post-hoc score fusion, since the context stream alone (71.94%) nearly matches the face stream alone (74.13%) on CAER and the joint model beats both.
Hiding the face during context training is an effective way to stop a context encoder from fixating on the most discriminative region, so it can discover secondary emotion cues.
Adaptive fusion lets the model lean on context when the face is ambiguous: the paper shows cases where the context weight $\lambda_C$ is above 0.7 for sad and fear clips and below 0.1 for happy clips.
Context-aware pretraining data transfers: training on CAER plus AFEW lifts AFEW accuracy from 41.86% (face-only) and 43.12% (AFEW only) to 51.68%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: the face-hiding step is never specified, so varying it (blackout, blur, inpainting, pixelation) and re-measuring the face-only-to-full accuracy gap would show whether the context gain is tied to one preprocessing choice.
Beyond the paper: the same hide-the-dominant-cue-and-attend scheme could transfer to other human-centric recognition tasks, such as group emotion or action recognition, whenever the face or body is the default discriminative feature and the scene is informative.
Beyond the paper: comparing the learned attention maps with human gaze data on the same video clips would test whether the context regions the network selects match the regions people use to judge emotion.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know one thing up front: the central empirical claim, that hiding faces and attending to context adds 2.91 points on CAER (Table 3), is not trustworthy as reported, because the random split is almost certainly not show-disjoint. The paper says the 13,201 clips, drawn from only 79 TV shows, are randomly split 70/10/20 with no mention of keeping shows apart. With multiple clips per show, the context stream can pick up show-specific color grading, set design, or genre-typical environments that correlate with the annotated emotion. The face-only stream cannot exploit those cues as easily, which would inflate the context gain. Figure 8 shows the context stream attends to scene regions, so it has the capacity. This is a load-bearing flaw, not a nitpick.

What the paper does well: the CAER dataset itself is a real contribution. 13,201 clips with confidence scores, seven categories, full frames plus face crops, and it is larger and more context-rich than AFEW or EMOTIC for this task. The two-stream architecture is straightforward and the ablations are clear: face-only 74.13, context-only 71.94, full model 77.04, and the fusion attention helps. The AFEW experiment in Table 6 is confounded (training on CAER+AFEW versus AFEW alone mixes extra data with context), but the authors honestly note they are not trying to beat ensemble methods like Fan et al. The paper is clearly written and the method is reproducible in principle.

Soft spots, in proportion: the face-hiding preprocessing is never described. Section 3.2.1 says inputs are "face-hidden" but not how, and the entire context stream depends on that. No code, no error bars anywhere, and results are single runs. The CAER filtering removed group and ambiguous clips, which is defensible for a clean benchmark but limits generalization to messy real video. The related work cites EMOTIC and Chen et al., so the novelty claim is appropriately scoped to the hiding strategy and the dataset.

If I were refereeing, I would ask for a show-disjoint split or at least an analysis of per-show accuracy, a specification of the face-hiding step, and standard errors over multiple runs. The dataset should still be published, and the method is worth testing on a properly split version. This deserves peer review, but the strong claim that context helps should be tempered until the leak is addressed.

Recommendation: send it to review, but flag the split issue as the first thing to fix. I would not cite the accuracy numbers yet, but I would cite the dataset if it becomes publicly available with a clean protocol.

Referee Report

4 major / 5 minor

Summary. The paper proposes CAER-Net, a two-stream network for emotion recognition from images and videos that combines a face stream with a context stream. The context stream is trained on face-hidden inputs and uses a learned spatial attention map to weight scene regions, and an adaptive fusion module learns per-sample weights to combine face and context features. The authors introduce the CAER/CAER-S benchmark of 13,201 TV-show clips with seven emotion labels and report ablations on CAER plus an evaluation on AFEW. The central claim is that jointly encoding facial expression and attention-weighted scene context improves emotion recognition accuracy over face-only models (77.04% vs 74.13% on CAER; 51.68% vs 41.86% on AFEW).

Significance. If the reported gains are robust, the paper provides a useful demonstration that scene context carries signal beyond facial expression and contributes a new large-scale benchmark that includes such context. The architecture is simple, the ablation structure directly targets the two-stream hypothesis, and the external AFEW evaluation is a valuable independent check. However, the main evidence is currently fragile: the CAER train/test split is at clip level and likely places clips from the same TV shows in both sets, all accuracies are single runs without error bars, and the face-hiding preprocessing is not specified. These issues do not invalidate the idea, but they must be addressed before the central claim is established.

major comments (4)

[Section 4.1 and Table 3] The CAER benchmark is said to be 'randomly split into training (70%), validation (10%), and testing (20%) sets,' without any show-disjoint constraint. Because the 13,201 clips come from 79 TV shows, a clip-level random split likely places clips from the same shows into both train and test. The context stream sees full frames (Section 5.1), so it can exploit show-specific color grading, set design, and genre-typical environments that correlate with the annotated emotion, while the face stream cannot. The 2.91-point gap between CAER-Net w/F and full CAER-Net may therefore reflect dataset leakage rather than emotion-relevant context. Please report a show-disjoint split, or quantify the number of shows appearing in both train and test, and rerun the Table 3 ablations under that protocol.
[Section 3.2.1] The paper repeatedly states that the context stream is trained on 'face-hidden inputs V_C' and that hiding faces is the key idea, but it never describes how faces are detected, masked, filled, or how the hiding is made temporally consistent across frames. It is also not stated whether the same hiding procedure is applied at test time. This is a load-bearing implementation detail: different hiding choices can leak facial information or remove scene structure, either of which changes the meaning of the reported ablation. Please specify the exact hiding algorithm and confirm that it is identical at train and test time.
[Section 5.3, Tables 3 and 6] All reported accuracies are single point estimates with no standard deviations, number of seeds, or statistical significance tests. Several conclusions rest on small differences, such as the 74.94% vs 75.57% comparison for adding fusion attention and the 2.91-point gain of the full model on CAER. Given the stochasticity of training 3D CNNs from scratch, these differences may be within run-to-run variation. Please report mean and standard deviation over at least three training runs, or otherwise provide significance evidence, for the main comparisons.
[Section 5.4, Table 6] The AFEW experiment confounds the effect of context with the effect of additional training data. 'CAER-Net AFEW' (43.12%) and 'CAER-Net w/F AFEW' (41.86%) are trained on AFEW only, while 'CAER-Net CAER+AFEW' (51.68%) is trained on both. The large gain therefore does not isolate the context stream's contribution. A controlled comparison needs at least a face-only CAER-Net trained on CAER+AFEW, and ideally a context-only model on AFEW alone, so that the training-data mix is held fixed when comparing face-only with face-plus-context.

minor comments (5)

[Section 3.2.1] The text says 'the number of filters for five convolution layers are 32, 64, 128, and 256, respectively,' which lists only four values for five layers; this should be corrected.
[Section 4.1] The filtering criteria for 'group-level and ambiguous shots' and the confidence scores used to remove clips are not defined operationally; please clarify how these were determined so that the benchmark construction is reproducible.
[Section 5.1 and Figure 8] The paper mentions an ablation 'without hiding the face during training,' but Table 3 does not include a row for this condition; please add the corresponding quantitative result or remove the claim.
[Section 3.2.1] The attention is said to be 'implicitly learned in an unsupervised manner,' but the attention module is trained end-to-end with emotion labels; 'learned implicitly from the classification loss' would be more accurate.
[Section 5.1] No code or implementation of the face-hiding step is provided; since the dataset URL is given, please also release a reference implementation and evaluation script to improve reproducibility.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central context-gain claim is grounded in direct ablations and external AFEW evaluation, with only a minor non-load-bearing self-citation.

full rationale

The paper's central claim is that adding an attention-weighted context stream to a face-expression stream improves emotion recognition. This claim is supported by Table 3, where CAER-Net with both streams reaches 77.04% versus 74.13% for the face-only stream on the newly introduced CAER benchmark, and by Table 6, where CAER-Net trained on CAER+AFEW reaches 51.68% on the external AFEW benchmark. The core comparison is an empirical ablation, not a derivation: the context stream is trained with a separate loss on face-hidden inputs, and the reported accuracy is measured on held-out test clips. No equation in the paper defines the predicted accuracy as equal to a fitted parameter or to an input constant. The only apparent self-citation is reference [20], an earlier ICASSP paper by overlapping authors, used when stating that the face encoding stream follows existing facial-expression-analysis approaches. That citation is architectural lineage and does not supply the central claim that scene context improves accuracy. The paper's evaluation on AFEW is external ground truth, and the internal ablation isolates the context and attention modules. The undocumented face-hiding preprocessing is a reproducibility gap and may affect results, but it is not a circularity: the model still learns to predict manually annotated emotion labels from visual inputs. Likewise, the possibility that the CAER train/test split is not show-disjoint is a data-leakage concern, not a case of the paper's conclusions being equivalent to its inputs by construction. The paper therefore exhibits no significant circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the reliability of automatic face detection and tracking, the chosen seven-category label space, the assumed sufficiency of face+context features, and several hand-picked training hyperparameters. No new physical entities are postulated. The annotation confidence threshold and clip sampling are the most consequential data-engineering choices.

free parameters (5)

Annotation confidence threshold = 0.5
Clips with average confidence under 0.5 were removed during dataset construction; this threshold shapes the benchmark and was chosen by hand.
Temporal clip length = 16 frames
Videos are divided into non-overlapping 16-frame clips for training; length is a design choice affecting temporal modeling.
Video sampling rate = 10 fps
Videos sampled at 10 frames per second before clipping; chosen by hand.
Learning rate schedule = 5e-3, drop 10x every 4 epochs
Hyperparameter selected for training CAER-Net from scratch; not fitted to data.
Dropout ratio = 0.5
Used between 1x1 convolution layers to prevent overfitting; chosen by hand.

assumptions (4)

domain assumption Seven discrete emotion categories (anger, disgust, fear, happy, sad, surprise, neutral) are mutually exclusive and exhaustive for the benchmark.
The dataset and model are built around this categorical label space, which may not capture compound or continuous emotions.
domain assumption Facial expression and scene context together are sufficient to infer emotion from a video clip.
The entire architecture assumes these two information sources carry the discriminative signal; audio is used only for annotation, not as model input.
domain assumption The face detector (dlib) and shot boundary detection/face tracking tools used to construct CAER are reliable enough for face cropping and hiding.
The face stream input and the face-hidden context input both depend on automatic detection/tracking; errors propagate into training.
domain assumption Spatial softmax attention over context features yields useful context localization.
The attention module is a learned architectural choice, not theoretically guaranteed to capture salient context.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Context-Aware Emotion Recognition Networks." pith.science (2026). https://pith.science/paper/2OVYUPSU

@misc{pith2026190805913,
  author       = {Pith},
  title        = {Pith review of: Context-Aware Emotion Recognition Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2OVYUPSU}},
  note         = {Machine review of arXiv:1908.05913}
}

read the original abstract

Traditional techniques for emotion recognition have focused on the facial expression analysis only, thus providing limited ability to encode context that comprehensively represents the emotional responses. We present deep networks for context-aware emotion recognition, called CAER-Net, that exploit not only human facial expression but also context information in a joint and boosting manner. The key idea is to hide human faces in a visual scene and seek other contexts based on an attention mechanism. Our networks consist of two sub-networks, including two-stream encoding networks to seperately extract the features of face and context regions, and adaptive fusion networks to fuse such features in an adaptive fashion. We also introduce a novel benchmark for context-aware emotion recognition, called CAER, that is more appropriate than existing benchmarks both qualitatively and quantitatively. On several benchmarks, CAER-Net proves the effect of context for emotion recognition. Our dataset is available at http://caer-dataset.github.io.

Figures

Figures reproduced from arXiv: 1908.05913 by the authors.

**Figure 2.** Network configuration of CAER-Net, consisting of two-stream encoding networks and adaptive fusion networks. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Visualization of the attention maps of (b) static and (c) [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 5.** Figure 5: Procedure for building CAER benchmark: we divide the [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

**Figure 6.** Figure 6: Examples in the EMOTIC [14], AffectNet [43] and CAER. While EMOTIC includes face-unvisible images to yeild ambiguous [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

**Figure 7.** Figure 7: Confusion matrix of CAER-Net with face stream only [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]

**Figure 8.** Figure 8: Visualization of the attention: (from top to bottom) in [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]

**Figure 10.** Figure 10: Visualization of learned attention maps in CAER-Net-S: (from top to bottom) inputs, attention maps of CAM [34], inputs of [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 50 canonical work pages

[1]

Toward an affect-sensitive autotutor

Sidney D’Mello, Rosalind W Picard, and Arthur Graesser. Toward an affect-sensitive autotutor. IEEE Int. Systems , 2007

work page 2007
[2]

Developing multimodal intelli- gent affective interfaces for tele-home health care

Christina Lisetti, Fatma Nasoz, Cynthia LeRouge, Onur Ozyer, and Kaye Alvarez. Developing multimodal intelli- gent affective interfaces for tele-home health care. Int. Jou. of Hum.-Comp. Stud., 2003

work page 2003
[3]

Experience- driven procedural content generation.IEEE Trans

Georgios N Yannakakis and Julian Togelius. Experience- driven procedural content generation.IEEE Trans. AC, 2011

work page 2011
[4]

Fa- cial expression recognition based on local binary patterns: A comprehensive study

Caifeng Shan, Shaogang Gong, and Peter W McOwan. Fa- cial expression recognition based on local binary patterns: A comprehensive study. Image and Vis. Comput., 2009

work page 2009
[5]

Learning active facial patches for expression analysis

Lin Zhong, Qingshan Liu, Peng Yang, Bo Liu, Junzhou Huang, and Dimitris N Metaxas. Learning active facial patches for expression analysis. In: CVPR, 2012

work page 2012
[6]

Emotionet: An accurate, real-time al- gorithm for the automatic annotation of a million facial ex- pressions in the wild

C Fabian Benitez-Quiroz, Ramprakash Srinivasan, and Aleix M Martinez. Emotionet: An accurate, real-time al- gorithm for the automatic annotation of a million facial ex- pressions in the wild. In: CVPR, 2016

work page 2016
[7]

Oc- clusion aware facial expression recognition using cnn with attention mechanism

Yong Li, Jiabei Zeng, Shiguang Shan, and Xilin Chen. Oc- clusion aware facial expression recognition using cnn with attention mechanism. IEEE Trans. IP, 2018

work page 2018
[8]

Reliable crowd- sourcing and deep locality-preserving learning for expres- sion recognition in the wild

Shan Li, Weihong Deng, and JunPing Du. Reliable crowd- sourcing and deep locality-preserving learning for expres- sion recognition in the wild. 2017

work page 2017

Show all 53 references

[9]

Acted facial expressions in the wild database

Abhinav Dhall, Roland Goecke, Simon Lucey, and Tom Gedeon. Acted facial expressions in the wild database. Tech- nical Report TR-CS-11, 2011

2011
[10]

Chal- lenges in representation learning: A report on three machine learning contests

Ian J Goodfellow, Dumitru Erhan, Pierre Luc Carrier, Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, Dong-Hyun Lee, et al. Chal- lenges in representation learning: A report on three machine learning contests. In: ICONIP, 2013

2013
[11]

Context in emotion perception

Lisa Feldman Barrett, Batja Mesquita, and Maria Gendron. Context in emotion perception. Curr . Dir . in Psych. Science, 2011

2011
[12]

The role of the parahippocampal cortex in cognition

Elissa M Aminoff, Kestutis Kveraga, and Moshe Bar. The role of the parahippocampal cortex in cognition. Trends in cognitive sciences, 2013

2013
[13]

Emotion in context: Deep semantic feature fusion for video emotion recognition

Chen Chen, Zuxuan Wu, and Yu-Gang Jiang. Emotion in context: Deep semantic feature fusion for video emotion recognition. In: MM, 2016

2016
[14]

Emotion recognition in context

Ronak Kosti, Jose M Alvarez, Adria Recasens, and Agata Lapedriza. Emotion recognition in context. In: CVPR, 2017

2017
[15]

Context-aware affective images classiﬁcation based on bi- layer sparse representation

Bing Li, Weihua Xiong, Weiming Hu, and Xinmiao Ding. Context-aware affective images classiﬁcation based on bi- layer sparse representation. In: MM, 2012

2012
[16]

Weakly supervised coupled networks for visual sentiment analysis

Jufeng Yang, Dongyu She, Yu-Kun Lai, Paul L Rosin, and Ming-Hsuan Yang. Weakly supervised coupled networks for visual sentiment analysis. In: CVPR, 2018

2018
[17]

Facial action coding system: a technique for the measurement of facial movement

E Friesen and Paul Ekman. Facial action coding system: a technique for the measurement of facial movement. Palo Alto, 1978

1978
[18]

Discriminative shared gaussian processes for multiview and view-invariant facial expression recognition.IEEE Trans

Stefanos Eleftheriadis, Ognjen Rudovic, and Maja Pantic. Discriminative shared gaussian processes for multiview and view-invariant facial expression recognition.IEEE Trans. IP, 2015

2015
[19]

Video-based emotion recognition using cnn-rnn and c3d hybrid networks

Yin Fan, Xiangju Lu, Dian Li, and Yuanliu Liu. Video-based emotion recognition using cnn-rnn and c3d hybrid networks. In: ICMI, 2016

2016
[20]

Spatiotemporal attention based deep neural networks for emotion recognition

Jiyoung Lee, Sunok Kim, Seungryong Kim, and Kwanghoon Sohn. Spatiotemporal attention based deep neural networks for emotion recognition. In: ICASSP, 2018

2018
[21]

Con- tinuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space

Mihalis A Nicolaou, Hatice Gunes, and Maja Pantic. Con- tinuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space. IEEE Trans. AC , 2011

2011
[22]

Recognizing emotions expressed by body pose: A biologi- cally inspired neural model

Konrad Schindler, Luc Van Gool, and Beatrice de Gelder. Recognizing emotions expressed by body pose: A biologi- cally inspired neural model. Neur . Net., 2008

2008
[23]

The extended cohn- kanade dataset (ck+): A complete dataset for action unit and emotion-speciﬁed expression

Patrick Lucey, Jeffrey F Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews. The extended cohn- kanade dataset (ck+): A complete dataset for action unit and emotion-speciﬁed expression. In: CVPR Work., 2010

2010
[24]

Web-based database for facial expression analysis.In: ICME, 2005

Maja Pantic, Michel Valstar, Ron Rademaker, and Ludo Maat. Web-based database for facial expression analysis.In: ICME, 2005

2005
[25]

Static facial expression analysis in tough condi- tions: Data, evaluation protocol and benchmark

Abhinav Dhall, Roland Goecke, Simon Lucey, and Tom Gedeon. Static facial expression analysis in tough condi- tions: Data, evaluation protocol and benchmark. In: ICCV Work., 2011

2011
[26]

Fa- cial expression recognition from world wild web

Ali Mollahosseini, Behzad Hasani, Michelle J Salvador, Ho- jjat Abdollahi, David Chan, and Mohammad H Mahoor. Fa- cial expression recognition from world wild web. In: CVPR Work., 2016

2016
[27]

Emotiw 2016: Video and group-level emotion recognition challenges

Abhinav Dhall, Roland Goecke, Jyoti Joshi, Jesse Hoey, and Tom Gedeon. Emotiw 2016: Video and group-level emotion recognition challenges. In: ICMI, 2016

2016
[28]

Patterson and J

G. Patterson and J. Hays. Coco attributes: Attributes for people, animals, and objects. In: ECCV, 2016

2016
[29]

Recog- nizing affective dimensions from body posture

Andrea Kleinsmith and Nadia Bianchi-Berthouze. Recog- nizing affective dimensions from body posture. In: ACII , 2007

2007
[30]

Kleinsmith, N

A. Kleinsmith, N. Bianchi-Berthouze, and A. Steed. Au- tomatic recognition of non-acted affective postures. IEEE Trans. Systems, 2011

2011
[31]

Imagenet classiﬁcation with deep convolutional neural net- works

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classiﬁcation with deep convolutional neural net- works. In: NeurIPS, 2012

2012
[32]

Very deep convo- lutional networks for large-scale image recognition

Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014

2014 arXiv
[33]

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In: CVPR , 2016

2016
[34]

Learning deep features for discrimi- native localization

Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discrimi- native localization. In: CVPR, 2016

2016
[35]

Grad-cam: Visual explanations from deep networks via gradient-based localization

Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In: ICCV, 2017

2017
[36]

Track and transfer: Watching videos to simulate strong human su- pervision for weakly-supervised object detection

Krishna Kumar Singh, Fanyi Xiao, and Yong Jae Lee. Track and transfer: Watching videos to simulate strong human su- pervision for weakly-supervised object detection. In: CVPR, 2016

2016
[37]

Cbam: Convolutional block attention module

Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. Cbam: Convolutional block attention module. In: ECCV, 2018

2018
[38]

Squeeze-and-excitation net- works

Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation net- works. In: CVPR, 2018

2018
[39]

Visual sentiment analysis by attending on local image regions

Quanzeng You, Hailin Jin, and Jiebo Luo. Visual sentiment analysis by attending on local image regions. In: AAAI , 2017

2017
[40]

Video- based emotion recognition using deeply-supervised neural networks

Yingruo Fan, Jacqueline CK Lam, and Victor OK Li. Video- based emotion recognition using deeply-supervised neural networks. In: ICMI, 2018

2018
[41]

Dlib-ml: A machine learning toolkit

Davis E King. Dlib-ml: A machine learning toolkit. Joul. of Mach. Learn. Res., 2009

2009
[42]

Ac- tion recognition using visual attention

Shikhar Sharma, Ryan Kiros, and Ruslan Salakhutdinov. Ac- tion recognition using visual attention. arXiv:1511.04119, 2015

2015 arXiv
[43]

Affectnet: A database for facial expression, valence, and arousal computing in the wild

Ali Mollahosseini, Behzad Hasani, and Mohammad H Ma- hoor. Affectnet: A database for facial expression, valence, and arousal computing in the wild. IEEE Trans. AC
[44]

Collecting large, richly annotated facial-expression databases from movies

Abhinav Dhall, Roland Goecke, Simon Lucey, Tom Gedeon, et al. Collecting large, richly annotated facial-expression databases from movies. IEEE Multi., 2012

2012
[45]

Predicting emotions in user-generated videos

Yu-Gang Jiang, Baohan Xu, and Xiangyang Xue. Predicting emotions in user-generated videos. In: AAAI, 2014

2014
[46]

Afew-va database for valence and arousal estimation in-the-wild

Jean Kossaiﬁ, Georgios Tzimiropoulos, Sinisa Todorovic, and Maja Pantic. Afew-va database for valence and arousal estimation in-the-wild. Image and Vis. Comput., 2017

2017
[47]

Automatic differentiation in pytorch

Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Al- ban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017

2017
[48]

Uniﬁed conﬁdence estimation net- works for robust stereo matching

Sunok Kim, Dongbo Min, Seungryong Kim, and Kwanghoon Sohn. Uniﬁed conﬁdence estimation net- works for robust stereo matching. IEEE Trans. IP, 2018

2018
[49]

Learning spatiotemporal features with 3d convolutional networks

Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learning spatiotemporal features with 3d convolutional networks. In: ICCV, 2015

2015
[50]

Imagenet: A large-scale hierarchical image database

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In: CVPR, 2009

2009
[51]

Large-scale video classiﬁcation with convolutional neural networks.In: CVPR, 2014

Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, and Li Fei-Fei. Large-scale video classiﬁcation with convolutional neural networks.In: CVPR, 2014

2014
[52]

Tem- poral multimodal fusion for video emotion classiﬁcation in the wild

Valentin Vielzeuf, St´ephane Pateux, and Fr´ed´eric Jurie. Tem- poral multimodal fusion for video emotion classiﬁcation in the wild. In: ICMI, 2017

2017
[53]

Learning supervised scoring ensemble for emotion recognition in the wild

Ping Hu, Dongqi Cai, Shandong Wang, Anbang Yao, and Yurong Chen. Learning supervised scoring ensemble for emotion recognition in the wild. In: ICMI, 2017

2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Toward an affect-sensitive autotutor

Sidney D’Mello, Rosalind W Picard, and Arthur Graesser. Toward an affect-sensitive autotutor. IEEE Int. Systems , 2007

work page 2007

[2] [2]

Developing multimodal intelli- gent affective interfaces for tele-home health care

Christina Lisetti, Fatma Nasoz, Cynthia LeRouge, Onur Ozyer, and Kaye Alvarez. Developing multimodal intelli- gent affective interfaces for tele-home health care. Int. Jou. of Hum.-Comp. Stud., 2003

work page 2003

[3] [3]

Experience- driven procedural content generation.IEEE Trans

Georgios N Yannakakis and Julian Togelius. Experience- driven procedural content generation.IEEE Trans. AC, 2011

work page 2011

[4] [4]

Fa- cial expression recognition based on local binary patterns: A comprehensive study

Caifeng Shan, Shaogang Gong, and Peter W McOwan. Fa- cial expression recognition based on local binary patterns: A comprehensive study. Image and Vis. Comput., 2009

work page 2009

[5] [5]

Learning active facial patches for expression analysis

Lin Zhong, Qingshan Liu, Peng Yang, Bo Liu, Junzhou Huang, and Dimitris N Metaxas. Learning active facial patches for expression analysis. In: CVPR, 2012

work page 2012

[6] [6]

Emotionet: An accurate, real-time al- gorithm for the automatic annotation of a million facial ex- pressions in the wild

C Fabian Benitez-Quiroz, Ramprakash Srinivasan, and Aleix M Martinez. Emotionet: An accurate, real-time al- gorithm for the automatic annotation of a million facial ex- pressions in the wild. In: CVPR, 2016

work page 2016

[7] [7]

Oc- clusion aware facial expression recognition using cnn with attention mechanism

Yong Li, Jiabei Zeng, Shiguang Shan, and Xilin Chen. Oc- clusion aware facial expression recognition using cnn with attention mechanism. IEEE Trans. IP, 2018

work page 2018

[8] [8]

Reliable crowd- sourcing and deep locality-preserving learning for expres- sion recognition in the wild

Shan Li, Weihong Deng, and JunPing Du. Reliable crowd- sourcing and deep locality-preserving learning for expres- sion recognition in the wild. 2017

work page 2017