Pith. sign in

REVIEW 4 major objections 6 minor 52 references

HeLo: Heterogeneous Multi-Modal Fusion with Label Correlation for Emotion Distribution Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that cross-attention fusion, optimal-transport alignment of physiological and behavioral features, and label-correlation-driven attention yield the best average rank across six emotion metrics on DMER and WESAD.

desk verdict Strong EDL results, but the printed forward pass doesn't type-check and the ground-truth correlation in Eq. (7) is degenerate; needs corrected equations and verified code before the claims can be trusted. read the letter →

arxiv 2507.06821 v3 pith:SKNJVUJP submitted 2025-07-09 cs.LG cs.AIcs.MM

Chuhang Zheng , Chunwei Tian , Jie Wen , Daoqiang Zhang , Qi Zhu This is my paper
classification cs.LGcs.AIcs.MM
keywords emotiondistributionlearningmulti-modalfusionoptimaltransportcross-attentionlabelcorrelationphysiologicalsignalsrecognition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Emotions rarely arrive one at a time: a person can feel nervous and afraid together, with each feeling at its own intensity. This paper is trying to establish that a learned model can predict such mixed emotion distributions better when it does three things at once: fuse the physiological signals (EEG, GSR, and PPG on DMER; ECG, EMG, and EDA on WESAD) using cross-attention; align the fused physiological representation with the behavioral data (facial video or accelerometer) through an optimal-transport matching flow; and use a learnable label embedding, whose correlation matrix is regularized toward the true label-correlation structure, to steer a final cross-attention layer. The claim is that this combination, called HeLo, attains the best average rank over six distribution-quality measures on two public datasets, DMER and WESAD, in both subject-dependent and subject-independent protocols. Accurate mixed-emotion prediction matters because it lets human-computer interaction respond to the mixture and intensity of real emotional experience rather than picking one dominant label.

What carries the argument

The load-bearing objects are three. (1) Cross-attention physiological fusion (CAPF): with EEG as the query and GSR and PPG as key–value pairs, multi-head cross-attention with residual connections produces the fused physiological representation, and making EEG the query lets the neural signal anchor the fusion. (2) The optimal-transport heterogeneity mining module (OTHM): a discrete Kantorovich optimal-transport problem defines the Wasserstein distance between the physiological and behavioral representations, and its optimal matching flow $T \in \mathbb{R}^{2C \times 2C}$, solved by a batch-wise Sinkhorn algorithm, is multiplied with the physiological representation before a transformer encoder, so the transport plan itself acts as a cross-modal correlation map that reduces distributional disparity. (3) The label correlation-driven cross-attention (LCDCA): a learnable label embedding $x_L \in \mathbb{R}^{l \times d}$ yields a learnable label-correlation matrix $M_L$ via cosine similarity, pulled toward the ground-truth label-correlation matrix $M_{gt}$ by an $\ell^2$ penalty; $M_L$ is then added inside the softmax logits of a cross-attention layer whose queries come from the label embedding and whose keys and values come from the multi-modal features, so semantic correlations among emotions directly shape the attended representation.

What would settle it

A reader could settle reproducibility by implementing the forward pass exactly as printed: multiplying the physiological representation of shape $2C \times d$ by the transport matrix $T$ of shape $2C \times 2C$ fails unless $d = 2C$ or a projection is inserted, and adding $M_L$ of shape $l \times l$ to $Q_L K_m^T$ of shape $l \times C'$ fails unless $C' = l$; if the published equations cannot be executed without hidden transposes or projections, the central empirical claim is not reproducible from the text. Running the released code and comparing its tensor shapes and its subject-independent WESAD average rank (reported as $1.16$) against the tables would settle whether the results correspond to the described architecture.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that explicit handling of two kinds of structure — cross-modal heterogeneity and cross-label correlation — pays off in emotion distribution learning. The authors propose HeLo, in which EEG is used as the query modality to attend over GSR and PPG features, giving a fused physiological representation $x_{Phy}$; a discrete optimal-transport matching flow $T \in \mathbb{R}^{2C \times 2C}$, computed by a batch-wise Sinkhorn solver of the Kantorovich formulation, then aligns $x_{Phy}$ with the behavioral representation $x_v$, and the transported features concatenated with the transformer-encoded behavioral features form the multi-modal representation. A learnable label embedding $x_L$ produces a label-correlation matrix $M_L$, trained by an $\ell^2$ loss toward the ground-truth correlation matrix $M_{gt}$ computed from the label distributions, and the final cross-attention adds $M_L$ into the attention logits, $x_o = \mathrm{softmax}((Q_L K_m^T + M_L)/\sqrt{d}) V_m$, so that attention is steered by the semantic relationships among emotions. The paper reports that this model records the best average rank across the six metrics on both datasets in both protocols (best on all six metrics except KL on DMER under subject dependence and KL on WESAD under subject independence), and that its ablations show every module and every modality contributes positively.

Load-bearing premise

That the architecture actually runs as described: the printed formulas leave the tensor dimensions of the transport multiplication and of the label-correlation addition unspecified, so the entire reported comparison rests on an unstated projection or transpose that a reader cannot verify from the equations alone.

Editorial extensions

If this is right

  • If HeLo's results hold, emotion distribution learning can treat label correlations as an explicit inductive bias: the learned correlation matrices cluster 'afraid' with 'nervous' and 'scared' in the paper's visualization, matching human intuition about which emotions co-occur.
  • The optimal-transport matching flow works as an align-then-fuse operator for physiology and behavior, which suggests transport plans can serve as interpretable cross-modal correlation maps rather than only as distance penalties inside a loss.
  • The paper reads its subject-independent (leave-one-subject-out) results as evidence that the aligned features carry partly subject-invariant information, which matters for deploying emotion models on users never seen during training.
  • The ablation studies imply that no modality and no module is redundant: removing any of them degrades all six distribution metrics, so the reported gains come from the full composition of the three modules.
  • The reported efficiency numbers (about 11.7 million FLOPs and 4.0 million parameters) place HeLo below three of the four deep baselines it is compared against, so the claimed accuracy does not obviously come at high compute cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the optimal-transport module is stated generically over two representation sets, the same align-then-fuse pattern could transfer to other heterogeneous modality pairs, such as text with images or audio with physiology, wherever a transport plan between two feature clouds is computable.
  • Editorial inference: the label-correlation matrix is added directly into the attention logits, effectively acting as a differentiable prior over label semantics, so the same mechanism could be grafted onto any label-distribution prediction head, not just the emotion setting.
  • Editorial inference: the paper's learned correlation matrices vary across subjects and trials, which suggests treating the correlation matrix as a subject descriptor for personalization, or as a signal for detecting trials where the emotion labels are ambiguous.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes HeLo, a multi-modal emotion distribution learning framework that fuses physiological signals (EEG, GSR, PPG in DMER; ECG, EMG, EDA in WESAD) with behavioral signals (video in DMER; ACC in WESAD) through three modules: cross-attention-based physiological fusion (CAPF), optimal-transport-based heterogeneity mining (OTHM), and label-correlation-driven cross-attention (LCDCA). The central claim, stated in the abstract and Section 4.4, is that HeLo achieves the best average rank across six distribution metrics on the DMER and WESAD datasets in both subject-dependent and subject-independent protocols, outperforming ten comparison methods including PT-SVM, AA-KNN, SA-CPNN, Conformer, MAET, CARAT, LDL-LRR, TLRLDL, CAD, and EmotionDict. The paper includes ablation studies on components, modalities, attention heads, transformer depth, feature visualizations, and model efficiency. The code is released at a GitHub link.

Significance. If the reported results hold and the architecture is implementable as described, the paper would be a useful contribution to multi-modal emotion distribution learning: it combines optimal transport for modality alignment with label-correlation-guided attention, evaluates on two public datasets under two protocols with six metrics, and provides extensive ablations and qualitative visualizations. The released code is a positive aspect that can support reproducibility. I find no circularity in the derivation: L_CC is an ordinary supervised target computed from ground-truth label distributions, not an assumption baked into the architecture. However, the significance is currently undercut by specification errors in the core equations (Eqs. (6), (7), and (9)), which prevent reproduction of the forward pass from the text alone, and by the absence of any statistical assessment of the reported superiority.

major comments (4)
  1. [Sec. 3.2, Eq. (6)] As written, Eq. (6) computes x_Phy ⊗ T, where x_Phy ∈ R^{2C×d} is multiplied on the right by T ∈ R^{2C×2C}. This matrix product is only defined if d = 2C, which is never stated and is inconsistent with the dataset feature dimensions reported in the supplementary material (e.g., 90 EEG, 28 GSR, 27 PPG, and 768 video features, with an embedding size of 128 given in Sec. 4.3). If the intended operation is instead T ⊗ x_Phy, the expression must be written with that order and with the projection from the raw behavioral features to R^{2C×d} explicitly defined. Without this correction, the OTHM module cannot be implemented from the text, and the reported gains cannot be attributed to the described mechanism.
  2. [Sec. 3.3, Eq. (9)] In Eq. (9), Q_L(K_m)^T is an l × C' matrix, where C' is the number of tokens in the projected multi-modal features x_m, while M_L is l × l. Adding these two matrices requires C' = l, but no projection from the multi-modal representation to exactly l tokens is described anywhere in Section 3. The text must specify the token count and the linear projections used to obtain K_m and V_m, otherwise the LCDCA forward pass is not reproducible from the paper.
  3. [Sec. 3.3, Eq. (7)] With L ∈ R^{l×1}, the entries L_i and L_j in Eq. (7) are nonnegative scalar label intensities. For any pair of nonzero intensities, the cosine similarity L_i · L_j / (||L_i|| · ||L_j||) equals 1, and it is undefined when either intensity is zero. The ground-truth correlation matrix M_gt is therefore either the all-ones matrix or undefined, which makes the correlation-constrained loss L_CC vacuous and contradicts the meaningful label correlations displayed in Figure 9 (e.g., 'afraid' correlated with 'nervous' and 'scared'). A meaningful ground-truth label correlation must be defined differently, for example through co-occurrence statistics over training samples or through correlations of full label-distribution vectors.
  4. [Sec. 4.3-4.5, Tables 1-2] The empirical claim of superiority rests entirely on point estimates. Tables 1 and 2 report single numbers per metric without standard deviations, confidence intervals, or significance tests. This is especially problematic in the subject-independent DMER results, where the margins over the second-best method are small (e.g., Chebyshev 0.0882 vs. 0.0925, Cosine 0.9148 vs. 0.9036). Moreover, the ablations over attention heads and transformer depth (Figures 6-7) appear to be conducted on the test set, because no validation split is described in Sec. 4.3. Please report variance across subject folds, perform paired significance tests across subjects, and state explicitly how hyperparameters were selected.
minor comments (6)
  1. [Sec. 4.4] The text says the methods are compared 'on both the DMER and AMIGOS datasets,' but the experiments use DMER and WESAD; AMIGOS is not mentioned anywhere else in the paper.
  2. [Table 1, DMER Canberra row] The ranks for Conformer (1.2581) and TLRLDL (1.3271) are both printed as 5; the correct ranks are 4 and 5 (or ties must be handled explicitly).
  3. [Table 1, DMER Intersection row] The ranks for TLRLDL (0.8537) and EmotionDict (0.8492) are both printed as 5; the correct ranks are 4 and 5.
  4. [Sec. 4.6, Figure 9 caption] The caption states that the learned label correlation M_L is visualized 'in Figure 7,' but the figure is numbered Figure 9.
  5. [Sec. 2.2 heading] The heading 'Muli-modal Emotion Recognition' contains a typo; it should be 'Multi-modal Emotion Recognition.'
  6. [Sec. 3.2, Eq. (5)] The minimization in Eq. (5) is written as 'min_{f∈Π(u,v)}' but the objective depends on T, not f; the optimization variable should be T to match the text and the definition of Π(u,v).

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: HeLo's reported gains are empirical test-set results; self-citations are contextual, while undefined tensor shapes and a degenerate ground-truth correlation are reproducibility issues, not circularity.

full rationale

The paper's claimed derivation chain is not circular. Input features are projected and fused by cross-attention (Eqs. 1-4); an OT flow T is solved from the physiological and behavioral representations themselves (Eq. 5) and used as a multiplicative interaction in Eq. 6; label embeddings x_L are trained with L_CC = ||M_L - M_gt||_2^2 (Eqs. 7-8), where M_gt is computed from the ground-truth label distribution L; and the final prediction L_hat is produced by a softmax MLP head and evaluated with held-out test-set metrics in Tables 1-2. Nothing in this chain is a fitted parameter renamed as a prediction: L_CC and KLD both use L as a supervised training target, which is ordinary supervised learning, not circular reasoning. The self-citations [24,49,51,52] are contextual references to prior multi-modal emotion-recognition work and do not supply any load-bearing uniqueness claim or ansatz for HeLo. The genuine problems in the paper are correctness and reproducibility issues, not circularity: Eq. (6) multiplies x_Phy of shape 2C x d against T of shape 2C x 2C without stating d = 2C or a projection, Eq. (9) adds an l x C' attention logit to an l x l matrix M_L without stating C' = l, and Eq. (7) as written with scalar nonnegative L_i would make M_gt all ones (or undefined at zeros), rendering L_CC vacuous. These defects mean the reported gains cannot be fully traced to the method as printed, but they are not a reduction of the output to the input by construction, so they do not raise the circularity score.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No new physical entities, particles, forces, or conserved quantities are introduced. The 'learnable label embedding' is a standard parameterized module, not a new entity with independent evidence. The ledger focuses on hyperparameters selected by test-set performance and on the domain assumptions behind the label correlation and OT designs.

free parameters (5)
  • number of attention heads = 4
    Selected by test-set performance on DMER (Figure 6); 'optimal performance across all six evaluation metrics when the number of attention heads was 4'.
  • transformer block depth = 1
    Selected by test-set performance on DMER (Figure 7); deeper blocks overfit.
  • Sinkhorn regularization parameter (epsilon) = unstated
    The OT module uses the Sinkhorn algorithm [4] but the entropic regularization strength is not reported; this affects the transport plan T and thus the fused features.
  • embedding and FFN dimensions = 128 and 64
    Implementation details state embedding size 128 and FFN size 64; values are hand-set.
  • learning rate, batch size, epochs = 1e-3, 128, 300
    Adam hyperparameters reported as fixed; no sensitivity analysis.
assumptions (5)
  • domain assumption Cosine similarity of label intensity vectors across a batch is a meaningful and stable target for label correlation learning.
    Used in Eq. (7)-(8) to define M_gt and M_L; the paper does not validate this choice against other correlation measures.
  • ad hoc to paper The optimal transport plan T between 2C physiological samples and 2C behavioral samples is a useful 'cross-modal correlation' to multiply into the features.
    Section 3.2, Eq. (6). No theoretical or empirical justification beyond the qualitative t-SNE visual in Figure 8.
  • standard math The Sinkhorn algorithm converges and the batch-wise implementation preserves differentiability.
    Section 3.2 relies on [4]; standard in OT literature.
  • domain assumption The feature extraction pipelines (DE features for EEG, LBP-TOP for video, etc.) are comparable across baselines.
    Supplementary A; baselines and HeLo use the same preprocessed features, but some baselines (EEG-only Conformer, text/image methods) may not be designed for these feature shapes.
  • domain assumption Ground-truth PANAS scores converted to emotion distributions are reliable supervision.
    Section 4.1; the label distributions are derived from self-reports after video viewing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HeLo: Heterogeneous Multi-Modal Fusion with Label Correlation for Emotion Distribution Learning." pith.science (2026). https://pith.science/paper/SKNJVUJP

@misc{pith2026250706821,
  author       = {Pith},
  title        = {Pith review of: HeLo: Heterogeneous Multi-Modal Fusion with Label Correlation for Emotion Distribution Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SKNJVUJP}},
  note         = {Machine review of arXiv:2507.06821}
}
read the original abstract

Multi-modal emotion recognition has garnered increasing attention as it plays a significant role in human-computer interaction (HCI) in recent years. Since different discrete emotions may exist at the same time, compared with single-class emotion recognition, emotion distribution learning (EDL) that identifies a mixture of basic emotions has gradually emerged as a trend. However, existing EDL methods face challenges in mining the heterogeneity among multiple modalities. Besides, rich semantic correlations across arbitrary basic emotions are not fully exploited. In this paper, we propose a multi-modal emotion distribution learning framework, named HeLo, aimed at fully exploring the heterogeneity and complementary information in multi-modal emotional data and label correlation within mixed basic emotions. Specifically, we first adopt cross-attention to effectively fuse the physiological data. Then, an optimal transport (OT)-based heterogeneity mining module is devised to mine the interaction and heterogeneity between the physiological and behavioral representations. To facilitate label correlation learning, we introduce a learnable label embedding optimized by correlation matrix alignment. Finally, the learnable label embeddings and label correlation matrices are integrated with the multi-modal representations through a novel label correlation-driven cross-attention mechanism for accurate emotion distribution learning. Experimental results on two publicly available datasets demonstrate the superiority of our proposed method in emotion distribution learning.

Figures

Figures reproduced from arXiv: 2507.06821 by the authors.

Figure 1
Figure 1. The challenges in emotion distribution learning [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The pipeline of the proposed HeLo. (a) Modality fusion stage: We first adopt a cross-attention mechanism to fuse the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the label correlation-driven cross [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Predicted emotion distributions of our model and comparison methods. We show two panels of two test samples of [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Ablation of network components in HeLo. increases, and subsequently declines. The model reaches its optimal performance across all six evaluation metrics when the number of attention heads was 4. However, the overall impact of the number of attention heads on the model…
Figure 6
Figure 6. Figure 6: Effect of attention heads on the DMER [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Effect of transformer block depth on the DMER. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Visualization of the feature embedding. 4.6 Visualization Visualization of the cross-modal representations. We visu￾alize the multi-modal feature space in the testing set by utilizing t-SNE [34]. In [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 49 canonical work pages

  1. [1]

    Soraia M Alarcao and Manuel J Fonseca. 2017. Emotions recognition using EEG signals: A survey.IEEE Transactions on Affective Computing10, 3 (2017), 374–393

    work page 2017

  2. [2]

    Yekta Said Can, Bhargavi Mahesh, and Elisabeth André. 2023. Approaches, applications, and challenges in physiological emotion recognition—a tutorial overview.Proc. IEEE(2023)

    work page 2023

  3. [3]

    Zongsheng Cao, Qianqian Xu, Zhiyong Yang, Yuan He, Xiaochun Cao, and Qingming Huang. 2022. Otkge: Multi-modal knowledge graph embeddings via optimal transport.Advances in Neural Information Processing Systems35 (2022), 39090–39102

    work page 2022

  4. [4]

    Liqun Chen, Zhe Gan, Yu Cheng, Linjie Li, Lawrence Carin, and Jingjing Liu. 2020. Graph optimal transport for cross-domain alignment. InInternational Conference on Machine Learning. PMLR, 1542–1553

    work page 2020

  5. [5]

    Bo Cheng and Guangyuan Liu. 2008. Emotion recognition from surface EMG signal using wavelet transform and neural network. In2008 2nd International Conference on Bioinformatics and Biomedical Engineering. IEEE, 1363–1366

    work page 2008

  6. [6]

    Bin-Bin Gao, Chao Xing, Chen-Wei Xie, Jianxin Wu, and Xin Geng. 2017. Deep label distribution learning with label ambiguity.IEEE Transactions on Image Processing26, 6 (2017), 2825–2838

    work page 2017

  7. [7]

    Xin Geng. 2016. Label distribution learning.IEEE Transactions on Knowledge and Data Engineering28, 7 (2016), 1734–1748

    work page 2016

  8. [8]

    Xin Geng and Rongzi Ji. 2013. Label Distribution Learning. In2013 IEEE 13th International Conference on Data Mining Workshops. IEEE, 377–383

    work page 2013

Show all 52 references
  1. [9]

    Xin Geng, Kate Smith-Miles, and Zhi-Hua Zhou. 2010. Facial Age Estimation by Learning from Label Distributions. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 24. 451–456

    2010

  2. [10]

    Xin Geng, Qin Wang, and Yu Xia. 2014. Facial age estimation by adaptive label distribution learning. In2014 22nd International Conference on Pattern Recognition. IEEE, 4465–4470

    2014

  3. [11]

    Xin Geng, Chao Yin, and Zhi-Hua Zhou. 2013. Facial age estimation by learning from label distributions.IEEE Transactions on Pattern Analysis and Machine Intelligence35, 10 (2013), 2401–2412

    2013

  4. [12]

    Yu-Liang Hsu, Jeen-Shing Wang, Wei-Chun Chiang, and Chien-Han Hung. 2017. Automatic ECG-based emotion recognition in music listening.IEEE Transactions on Affective Computing11, 1 (2017), 85–99

    2017

  5. [13]

    Xiuyi Jia, Xiaoxia Shen, Weiwei Li, Yunan Lu, and Jihua Zhu. 2023. Label Distri- bution Learning by Maintaining Label Ranking Relation.IEEE Transactions on Knowledge & Data Engineering35, 02 (2023), 1695–1707

    2023

  6. [14]

    Xiuyi Jia, Xiang Zheng, Weiwei Li, Changqing Zhang, and Zechao Li. 2019. Facial emotion distribution learning by exploiting low-rank label correlations locally. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9841–9850

    2019

  7. [15]

    Wei-Bang Jiang, Xuan-Hao Liu, Wei-Long Zheng, and Bao-Liang Lu. 2023. Multi- modal adaptive emotion transformer with flexible modality inputs on a novel dataset with continuous labels. InProceedings of the 31st ACM International Conference on Multimedia. 5975–5984

    2023

  8. [16]

    Leonid V Kantorovich. 2006. On the Translocation of Masses.Journal of Mathe- matical Sciences133, 4 (2006)

    2006

  9. [17]

    Jonghwa Kim and Elisabeth André. 2008. Emotion recognition based on phys- iological changes in music listening.IEEE transactions on pattern analysis and machine intelligence30, 12 (2008), 2067–2083

    2008

  10. [18]

    Sander Koelstra, Christian Muhl, Mohammad Soleymani, Jong-Seok Lee, Ashkan Yazdani, Touradj Ebrahimi, Thierry Pun, Anton Nijholt, and Ioannis Patras. 2011. Deap: A database for emotion analysis; using physiological signals.IEEE transac- tions on affective computing3, 1 (2011), 18–31

    2011

  11. [19]

    Zhiqiang Kou, Jing Wang, Jiawei Tang, Yuheng Jia, Boyu Shi, and Xin Geng. 2024. Exploiting Multi-Label Correlation in Label Distribution Learning. InProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI- 24, Kate Larson (Ed.). Interna...

    2024 doi

  12. [20]

    Peter J Lang. 1995. The emotion probe: Studies of motivation and attention. American psychologist50, 5 (1995), 372

    1995

  13. [21]

    Cunbo Li, Peiyang Li, Yangsong Zhang, Ning Li, Yajing Si, Fali Li, Zehong Cao, Huafu Chen, Badong Chen, Dezhong Yao, et al. 2023. Effective emotion recogni- tion by learning discriminative graph topologies in EEG brain networks.IEEE Transactions on Neural Networks and Learning...

    2023

  14. [22]

    Shan Li and Weihong Deng. 2019. Blended emotion in-the-wild: Multi-label facial expression recognition using crowdsourced annotations and deep locality feature learning.International Journal of Computer Vision127, 6 (2019), 884–906

    2019

  15. [23]

    Fang Liu, Pei Yang, Yezhi Shu, Fei Yan, Guanhua Zhang, and Yong-Jin Liu. 2023. Emotion dictionary learning with modality attentions for mixed emotion explo- ration.IEEE Transactions on Affective Computing(2023)

    2023

  16. [24]

    Minxu Liu, Donghai Guan, Chuhang Zheng, and Qi Zhu. 2025. Multi-Modal Discriminative Network for Emotion Recognition across Individuals.IEEE Trans- actions on Cognitive and Developmental Systems(2025), 1–13. doi:10.1109/TCDS. 2025.3552124

    2025

  17. [25]

    Wang Kay Ngai, Haoran Xie, Di Zou, and Kee-Lee Chou. 2022. Emotion recogni- tion based on convolutional neural networks and heterogeneous bio-signal data sources.Information Fusion77 (2022), 107–117

    2022

  18. [26]

    Cheng Peng, Ke Chen, Lidan Shou, and Gang Chen. 2024. CARAT: Contrastive Feature Reconstruction and Aggregation for Multi-Modal Multi-Label Emotion Recognition. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 14581–14589

    2024

  19. [27]

    Michaël Perrot, Nicolas Courty, Rémi Flamary, and Amaury Habrard. 2016. Map- ping estimation for discrete optimal transport.Advances in Neural Information Processing Systems29 (2016)

    2016

  20. [28]

    Philip Schmidt, Attila Reiss, Robert Duerichen, Claus Marberger, and Kristof Van Laerhoven. 2018. Introducing wesad, a multimodal dataset for wearable stress and affect detection. InProceedings of the 20th ACM International Conference on Multimodal Interaction. 400–408

    2018

  21. [29]

    2010.Designing the user interface: strategies for effective human-computer interaction

    Ben Shneiderman and Catherine Plaisant. 2010.Designing the user interface: strategies for effective human-computer interaction. Pearson Education India

    2010

  22. [30]

    Yezhi Shu, Pei Yang, Niqi Liu, Shu Zhang, Guozhen Zhao, and Yong-Jin Liu. 2022. Emotion distribution learning based on peripheral physiological signals.IEEE Transactions on Affective Computing14, 3 (2022), 2470–2483

    2022

  23. [31]

    Andrew H Song, Richard J Chen, Guillaume Jaume, Anurag J Vaidya, Alexander S Baras, and Faisal Mahmood. 2024. Multimodal Prototyping for cancer survival prediction.arXiv preprint arXiv:2407.00224(2024)

    2024 arXiv

  24. [32]

    Yonghao Song, Qingqing Zheng, Bingchuan Liu, and Xiaorong Gao. 2022. EEG conformer: Convolutional transformer for EEG decoding and visualization.IEEE Transactions on Neural Systems and Rehabilitation Engineering31 (2022), 710–719

    2022

  25. [33]

    Goran Udovičić, Jurica Ðerek, Mladen Russo, and Marjan Sikora. 2017. Wearable emotion recognition system based on GSR and PPG signals. InProceedings of the 2nd international workshop on multimedia for personal health and health care. 53–59

    2017

  26. [34]

    Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research9, 11 (2008)

    2008

  27. [35]

    Monu Verma, M Satish Kumar Reddy, Yashwanth Reddy Meedimale, Murari Mandal, and Santosh Kumar Vipparthi. 2021. Automer: Spatiotemporal neural architecture search for microexpression recognition.IEEE Transactions on Neural Networks and Learning Systems33, 11 (2021), 6116–6128

    2021

  28. [36]

    D Watson. 1994. The PANAS-X: Manual for the positive and negative affect schedule-expanded form.The University of Iowa(1994)

    1994

  29. [37]

    Changsong Wen, Xin Zhang, Xingxu Yao, and Jufeng Yang. 2023. Ordinal label distribution learning. InProceedings of the IEEE/CVF International Conference on Computer Vision. 23481–23491

    2023

  30. [38]

    Patti Williams and Jennifer L Aaker. 2002. Can mixed emotions peacefully coexist?Journal of consumer research28, 4 (2002), 636–649

    2002

  31. [39]

    Wen-Jing Yan, Xiaobai Li, Su-Jing Wang, Guoying Zhao, Yong-Jin Liu, Yu-Hsin Chen, and Xiaolan Fu. 2014. CASME II: An improved spontaneous micro- expression database and the baseline evaluation.PloS one9, 1 (2014), e86041

    2014

  32. [40]

    Pei Yang, Niqi Liu, Xinge Liu, Yezhi Shu, Wenqi Ji, Ziqi Ren, Jenny Sheng, Minjing Yu, Ran Yi, Dan Zhang, et al. 2024. A Multimodal Dataset for Mixed Emotion Recognition.Scientific Data11, 1 (2024), 847

    2024

  33. [41]

    Mengqing Ye, CL Philip Chen, and Tong Zhang. 2022. Hierarchical dynamic graph convolutional network with interpretability for EEG-based emotion recognition. IEEE Transactions on Neural Networks and Learning Systems(2022)

    2022

  34. [42]

    Lu Yi and Man-Wai Mak. 2020. Improving speech emotion recognition with adversarial data augmentation network.IEEE Transactions on Neural Networks and Learning Systems33, 1 (2020), 172–184

    2020

  35. [43]

    Dong Zhang, Xincheng Ju, Junhui Li, Shoushan Li, Qiaoming Zhu, and Guodong Zhou. 2020. Multi-modal multi-label emotion detection with modality and label dependence. InProceedings of the 2020 Conference on Empirical Methods in Natural Language processing (EMNLP). 3584–3593

    2020

  36. [44]

    Dong Zhang, Xincheng Ju, Wei Zhang, Junhui Li, Shoushan Li, Qiaoming Zhu, and Guodong Zhou. 2021. Multi-modal multi-label emotion recognition with het- erogeneous hierarchical message passing. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 14338–14346

    2021

  37. [45]

    Yuxiang Zhang, Jiamei Fu, Dongyu She, Ying Zhang, Senzhang Wang, and Jufeng Yang. 2018. Text Emotion Distribution Learning via Multi-Task Convolutional Neural Network.. InIJCAI. 4595–4601

    2018

  38. [46]

    Yulin Zhang, Guozhen Zhao, Yezhi Shu, Yan Ge, Dan Zhang, Yong-Jin Liu, and Xianghong Sun. 2021. CPED: A Chinese positive emotion database for emotion elicitation and analysis.IEEE Transactions on Affective Computing14, 2 (2021), 1417–1430

    2021

  39. [47]

    Guoying Zhao and Matti Pietikainen. 2007. Dynamic texture recognition using local binary patterns with an application to facial expressions.IEEE transactions on pattern analysis and machine intelligence29, 6 (2007), 915–928

    2007

  40. [48]

    Guozhen Zhao, Yulin Zhang, Guanhua Zhang, Dan Zhang, and Yong-Jin Liu. 2020. Multi-target positive emotion recognition from EEG signals.IEEE Transactions on Affective Computing14, 1 (2020), 370–381

    2020

  41. [49]

    Chuhang Zheng, Wei Shao, Daoqiang Zhang, and Qi Zhu. 2023. Prior-driven dynamic brain networks for multi-modal emotion recognition. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention. Conference’17, July 2017, Washington, DC, USA Chuhang Z...

    2023

  42. [50]

    Wei-Long Zheng and Bao-Liang Lu. 2016. Personalizing EEG-based affective models with transfer learning. InProceedings of the twenty-fifth International Joint Conference on Artificial Intelligence. 2732–2738

    2016

  43. [51]

    Qi Zhu, Chuhang Zheng, Zheng Zhang, Wei Shao, and Daoqiang Zhang. 2024. Dy- namic Confidence-Aware Multi-Modal Emotion Recognition.IEEE Transactions on Affective Computing15, 3 (2024), 1358–1370. doi:10.1109/TAFFC.2023.3340924

    2024

  44. [52]

    Qi Zhu, Ting Zhu, Lunke Fei, Chuhang Zheng, Wei Shao, David Zhang, and Daoqiang Zhang. 2025. Multi-Modal Cross-Subject Emotion Feature Alignment and Recognition with EEG and Eye Movements.IEEE Transactions on Affective Computing(2025), 1–15. doi:10.1109/TAFFC.2025.3554399 HeLo...

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.