Pith. sign in

REVIEW 9 cited by

Face Behavior a la carte: Expressions, Affect and Action Units in a Single Network

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1910.11111 v3 pith:Z2A2KJFB submitted 2019-10-15 cs.CV cs.HCcs.LGstat.ML

Dimitrios Kollias , Viktoriia Sharmanska , Stefanos Zafeiriou This is my paper
classification cs.CVcs.HCcs.LGstat.ML
keywords tasksfacialautomaticbehaviorbehaviournetworktrainingaction
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Automatic facial behavior analysis has a long history of studies in the intersection of computer vision, physiology and psychology. However it is only recently, with the collection of large-scale datasets and powerful machine learning methods such as deep neural networks, that automatic facial behavior analysis started to thrive. Three of its iconic tasks are automatic recognition of basic expressions (e.g. happy, sad, surprised), estimation of continuous emotions (e.g., valence and arousal), and detection of facial action units (activations of e.g. upper/inner eyebrows, nose wrinkles). Up until now these tasks have been mostly studied independently collecting a dataset for the task. We present the first and the largest study of all facial behaviour tasks learned jointly in a single multi-task, multi-domain and multi-label network, which we call FaceBehaviorNet. For this we utilize all publicly available datasets in the community (around 5M images) that study facial behaviour tasks in-the-wild. We demonstrate that training jointly an end-to-end network for all tasks has consistently better performance than training each of the single-task networks. Furthermore, we propose two simple strategies for coupling the tasks during training, co-annotation and distribution matching, and show the advantages of this approach. Finally we show that FaceBehaviorNet has learned features that encapsulate all aspects of facial behaviour, and can be successfully applied to perform tasks (compound emotion recognition) beyond the ones that it has been trained in a zero- and few-shot learning setting.

Discussion (0). Sign in to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DVD: A Comprehensive Dataset for Advancing Violence Detection in Real-World Scenarios

    cs.CV 2025-05 conditional novelty 6.0 of 10

    The authors propose a new frame-level annotated violence detection dataset, DVD, with 500 videos and rich metadata, but it is not yet available and lacks validation experiments.

  2. AffectFlow-DINO: Uncertainty-Aware Multi-Task Affect Estimation via Conditional Rectified Flow

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Adding a conditional rectified-flow head to a DINOv3 multi-task affect model improves valence-arousal CCC by +0.058 when the backbone is frozen and, with fine-tuning and validation-tuned calibration, reaches P_MTL=1.1...

  3. Causal Supervision of Attention for Affective Behaviour Analysis

    cs.CV 2026-07 unverdicted novelty 5.0 of 10

    Causal supervision plus K-V independence and SwiGLU attention pooling yields a multi-task P-score of 1.2214 on s-Aff-Wild2 validation for valence-arousal, expression, and action-unit prediction.

  4. Strength-Parity Ensembling with Parameter-Isolated Experts for Multi-Task Affect Recognition

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Parameter-isolated LoRA experts on one face backbone stay decorrelated (0.91 vs 0.98 for full fine-tuning) and improve an ABAW affect ensemble from 1.6669 to 1.6949–1.7259 validation score.

  5. A Shared Latent for Partially-Labeled Multi-Task Facial Affect Recognition

    cs.CV 2026-07 accept novelty 5.0 of 10

    A shared variational affect latent that marginalizes missing labels lifts rare expression and action-unit recognition on s-Aff-Wild2 beyond masked-loss training.

  6. Distance-aware Soft Prompt Guidance for Multimodal Valence-Arousal Estimation

    cs.CV 2026-03 reject novelty 5.0 of 10

    Distance-aware soft prompts over a 3×3 emotion grid with CLIP text prototypes and audio-visual GRU fusion achieve CCC_mean 0.5361 on Aff-Wild2, beating only the paper's self-defined baselines.

  7. AffectFuse: Cross-Task Feature Fusion with Temporal Modeling for Multi-Task Affective Behavior Analysis

    cs.CV 2026-07 conditional novelty 4.0 of 10

    A multi-task affective system combining frozen AffectNet backbones, LoRA-adapted MAE for action units, temporal heads, fusion, and ensembling attains P=1.7302 on the s-Aff-Wild2 validation split.

  8. Multimodal Alignment with Cross-Attentive GRUs for Fine-Grained Video Understanding

    cs.CV 2025-07 reject novelty 4.0 of 10

    A GRU-based cross-attention fusion of frozen vision-language encoders is claimed to achieve strong results on DVD and Aff-Wild2, but the supporting experiments are missing from the paper.

  9. TAGF: Time-aware Gated Fusion for Multimodal Valence-Arousal Estimation

    cs.MM 2025-07 reject novelty 4.0 of 10

    TAGF adds a BiLSTM-based gate that reweights recursive cross-attention outputs for valence-arousal prediction, with results slightly below several existing methods on Aff-Wild2.

Pith tools