Pith. sign in

REVIEW 4 major objections 7 minor 3 cited by

Milmer: a Framework for Multiple Instance Learning based Multimodal Emotion Recognition

T0 review · 4 major / 7 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read EEG plus facial video pushes four-class emotion accuracy to 96.72%.

desk verdict Plausible engineering combination, but the 96.72% DEAP-4 result is not verifiable as reported because the evaluation protocol is underspecified and the token-size selection in Table 4 appears to be done on the test set. read the letter →

arxiv 2502.00547 v1 pith:TYXACEVV submitted 2025-02-01 cs.CV cs.AIcs.HC

Zaitian Wang , Jian He , Yu Liang , Xiyuan Hu , Tianhao Peng , Kaixin Wang , Jiakai Wang , Chenlong Zhang
show 3 more authors
Weili Zhang Shuang Niu Xiaoyang Xie
This is my paper · ORCID
classification cs.CVcs.AIcs.HC
keywords multimodalemotionrecognitionmultipleinstancelearningEEGsignalprocessingfacialexpressiontransformerfusioncross-attentionmechanismDEAPdatasetvalence-arousalclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that multimodal emotion recognition fails when the two modalities are fused too crudely or when facial expressions are compressed into one frame, and that both defects can be fixed with a transformer-based pipeline. It introduces Milmer, which pairs filtered EEG segments with a bag of facial frames, selects the most informative frames by attention-based multiple instance learning, compresses their visual tokens with cross-attention, and lets a transformer fuse the two modalities. On the DEAP dataset the framework reports 96.72% accuracy and 96.71% F1 in the four-class valence-arousal task, above the best prior result it cites (83.20% accuracy). The authors also report the strongest accuracy in the two- and three-class tasks and show in ablations that each module contributes.

What carries the argument

The machinery is a three-module pipeline: an EEG preprocessing stage (1-50 Hz bandpass filter plus Fast ICA artifact removal), a facial feature extraction and balancing stage, and a transformer fusion stage. The distinctive object is the attention-based multiple instance learning step: a bag of facial frames is weighted by AMIL attention scores, and the top-K frames are kept instead of pooling them to one vector, preserving temporal information for fusion. A cross-attention layer then compresses the many visual tokens to N learnable queries, balancing token counts between modalities before a transformer encoder with modal-type and position embeddings pools a CLS token for classification. All three modules are shown in ablation to contribute, with the MIL plus cross-attention combination giving the largest single jump.

What would settle it

Run the DEAP-4 experiment with a separate validation split used to pick the 64-token visual size, include all participants who have facial videos, and report accuracy under a strict subject-independent protocol; if the number drops materially below 96.72% or below the 83.20% of the cited prior method under the same protocol, the headline claim is protocol-bound rather than a property of the fusion design.

Watch

Extended reading notes

Core claim

Milmer's core claim is that emotion classification improves when facial expression videos are treated as bags of frames rather than single images, and when the resulting visual tokens are balanced against EEG tokens before fusion. For each 3-second segment, ten evenly spaced facial frames are encoded by a fine-tuned Swin Transformer, attention weights pick the top-K frames, and cross-attention compresses the M visual tokens into N learnable queries sized to match the EEG token count. A transformer fusion module with modal-type, positional, and CLS embeddings then produces the four-class valence-arousal prediction. The paper reports 96.72% accuracy and 96.71% F1 on DEAP-4, exceeding the best compared method by 13.52 accuracy points, and similar advantages on DEAP-3 and DEAP-2.

Load-bearing premise

The reported 96.72% depends on the evaluation being unbiased when hyperparameters such as the visual token size are chosen, and on excluding Participant 11 not materially changing the result.

Editorial extensions

If this is right

  • Four-class DEAP emotion recognition can be treated as a nearly saturated benchmark under the paper's subject-dependent evaluation, with accuracy over 96%.
  • Treating a video segment as a bag of frames, rather than picking one representative image, raises recognition accuracy even when the same backbone is used.
  • Fusion quality depends on balancing the token counts of heterogeneous modalities: shrinking visual tokens toward the EEG token count improves results up to a point, then hurts.
  • Transformer-based fusion outperforms concatenation and DeepCCA-style correlation fusion for EEG-plus-face emotion recognition by more than seven accuracy points.
  • A drop in accuracy under subject-independent evaluation shows the framework gains from intra-subject pattern learning, not only from general emotion features.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A strict subject-independent rerun is the natural check: the paper's own confusion-matrix analysis shows a gap, so the headline 96.72% should be read as subject-dependent performance.
  • The balancing idea is modular: the same cross-attention token-compression could apply to any pair of modalities with unequal token counts, such as audio-text fusion.
  • Testing on other synchronized video-physiology emotion datasets would show whether the gains are specific to the DEAP setup or structural to the bag-of-frames approach.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The manuscript proposes Milmer, a multimodal emotion recognition framework that fuses EEG signals and facial video frames on the DEAP dataset. The method uses a fine-tuned Swin Transformer for visual features, attention-based multiple instance learning with top-K frame selection, a cross-attention module that compresses the visual tokens, and a transformer fusion module with modality-type and position embeddings. The paper reports accuracy and F1 results for DEAP-2, DEAP-3, and DEAP-4, with the headline claim of 96.72% accuracy and 96.71% F1 on the four-class DEAP-4 task, together with ablation studies for the proposed modules.

Significance. If the reported accuracies were obtained under an unbiased evaluation protocol, the contribution would be significant for multimodal affective computing: the DEAP-4 result is more than 13 points above the strongest cited comparator, and the MIL plus cross-attention design addresses a real gap in using temporal facial information. The authors also promise to release code, which would aid reproducibility. However, the paper does not specify the train/validation/test split, and Table 4 indicates that the visual token size was selected as the best among several configurations without a described held-out validation set. Until the evaluation protocol is clarified and the selection procedure is shown to be unbiased, the numerical claims cannot be credited.

major comments (4)
  1. [Section 4.1] The train/validation/test split is never specified. The text states that data are segmented into 3-second windows and that each EEG window is paired with 10 facial frames, but it does not say whether the split is by subject, by trial, or random, nor which split is used for the headline numbers in Table 1. Because adjacent windows from the same 60-second trial are temporally correlated, a random window-level split can substantially inflate classification accuracy. This omission is load-bearing because the abstract's 96.72% claim and all subsequent comparisons are uninterpretable without knowing the protocol.
  2. [Table 4] The visual token output size N=64 is selected as the best among seven tested values (196, 147, 128, 96, 64, 32, 16), and the paper does not mention a separate validation set for this selection. If the test set was used to choose N, the reported 96.72% is optimistically biased. The authors should select hyperparameters on a held-out validation split and report test-set accuracy only for the chosen configuration, along with the validation-based selection criterion.
  3. [Section 4.1] Participant 11 is excluded because three facial expression videos are missing, and no robustness analysis is provided. Only 22 of the 32 DEAP participants have facial data, so excluding Participant 11 leaves 21 subjects; the paper does not assess how this exclusion affects the results, for example by rerunning with the remaining facial videos of Participant 11 or by reporting per-participant accuracy. Since the dataset is small, this exclusion could materially change the reported numbers.
  4. [Tables 5 and 6] The ablation results are reported without standard deviations, repeated runs, or a statement of the underlying split, so it is impossible to judge whether differences such as 96.72% versus 95.99% (Table 4) or 94.46% versus 96.72% (Table 6) are significant. Given that the model has multiple stochastic components, single-run accuracies without error bars are insufficient support for the claim that the full configuration is the best.
minor comments (7)
  1. [Section 5] The conclusion begins with an unrelated paragraph about assessing the credibility of Internet of Things devices, which appears to be a copy-paste error. This paragraph should be removed and replaced with a conclusion that accurately summarizes the emotion recognition work presented in the paper.
  2. [Abstract and Section 6] The sentence "Our code are available at ..." should be corrected to "Our code is available at ...".
  3. [Equation (5)] The labels "HAL V" and "LAL V" contain an awkward space and are inconsistent with the compact notation "HAHV" and "LAHV" used in the same equation; the formatting should be made uniform.
  4. [Section 1] The sentence beginning "Each modality contributes unique strengths..." is repeated verbatim in the introduction; one copy should be deleted.
  5. [References] Reference [13] cites ViLT, a vision-and-language pretraining model, but the surrounding text discusses multimodal emotion recognition fusion; a more directly relevant citation is needed.
  6. [Table 1] There is a typo in the first row: "Gupta wt al." should be "Gupta et al.".
  7. [Section 4.2] The phrase "experiments conduct under subject-dependent" should be "experiments conducted under subject-dependent".

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the empirical benchmark evaluation is self-contained; reported accuracy is measured, not derived.

full rationale

The paper's central claim is an empirical accuracy measurement on the DEAP benchmark, not a result derived from its inputs. The proposed architecture (Swin feature extraction, MIL top-K selection, cross-attention balancing, and transformer fusion) is specified by explicit equations (1)-(3) and evaluated against an external dataset, so the reported 96.72% DEAP-4 accuracy is a measured outcome rather than a quantity forced by definition. The self-citations [6] and [8] appear only as background references for EEG-based emotion analysis and are not load-bearing for the framework's validity. No uniqueness theorem, ansatz, or prior-work result is invoked to forbid alternatives or to make the architecture's choice necessary. The most salient concern is that Section 4 does not specify a trial-disjoint train/validation/test split and Table 4 selects the visual-token output size (64) from among several evaluated configurations, which is a potential evaluation-bias or soundness issue rather than a circular reasoning issue. Accordingly, no circular step can be exhibited, and the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The model's performance depends on several hand-chosen or test-set-tuned hyperparameters (N=64, unreported K, MLP dimension, classification threshold), as well as assumptions about DEAP ground truth and ICA artifact removal. These are not derived from first principles and limit the generality of the reported result.

free parameters (4)
  • Number of visual tokens N after cross-attention = 64
    Selected as the best among seven tested values (16, 32, 64, 96, 128, 147, 196) based on test set accuracy (Table 4); this is a test-set-tuned hyperparameter.
  • Top-K number of frames selected by MIL = not reported
    The paper does not report the value of K, the number of selected frames, yet it is a critical design choice that affects performance.
  • MLP upsampling dimension = 768
    EEG signals (384 samples) are upsampled to 768 to align with image data; this dimension is chosen by hand without ablation.
  • Classification threshold = 5
    The 4-class DEAP split uses a fixed threshold of 5 on continuous ratings; this is a dataset-specific choice.
assumptions (4)
  • domain assumption DEAP self-reported ratings are reliable ground truth for emotion labels
    The paper uses the participant ratings on valence and arousal as ground truth, which assumes the ratings accurately reflect emotional state.
  • domain assumption ICA artifact removal with default MNE settings sufficiently cleans EEG signals
    The paper applies Fast ICA with default settings and assumes this removes ocular and muscular artifacts without removing relevant signal.
  • ad hoc to paper Swin Transformer fine-tuned on an unspecified emotion dataset transfers to DEAP facial frames
    The paper states the Swin backbone is initialized with weights fine-tuned on emotion-related datasets but does not specify which dataset or fine-tuning procedure, making the transfer an unverified assumption.
  • domain assumption The 3-second window with 10 evenly spaced frames captures the emotional dynamics
    This temporal sampling choice is made without supporting analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Milmer: a Framework for Multiple Instance Learning based Multimodal Emotion Recognition." pith.science (2026). https://pith.science/paper/TYXACEVV

@misc{pith2026250200547,
  author       = {Pith},
  title        = {Pith review of: Milmer: a Framework for Multiple Instance Learning based Multimodal Emotion Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TYXACEVV}},
  note         = {Machine review of arXiv:2502.00547}
}
read the original abstract

Emotions play a crucial role in human behavior and decision-making, making emotion recognition a key area of interest in human-computer interaction (HCI). This study addresses the challenges of emotion recognition by integrating facial expression analysis with electroencephalogram (EEG) signals, introducing a novel multimodal framework-Milmer. The proposed framework employs a transformer-based fusion approach to effectively integrate visual and physiological modalities. It consists of an EEG preprocessing module, a facial feature extraction and balancing module, and a cross-modal fusion module. To enhance visual feature extraction, we fine-tune a pre-trained Swin Transformer on emotion-related datasets. Additionally, a cross-attention mechanism is introduced to balance token representation across modalities, ensuring effective feature integration. A key innovation of this work is the adoption of a multiple instance learning (MIL) approach, which extracts meaningful information from multiple facial expression images over time, capturing critical temporal dynamics often overlooked in previous studies. Extensive experiments conducted on the DEAP dataset demonstrate the superiority of the proposed framework, achieving a classification accuracy of 96.72% in the four-class emotion recognition task. Ablation studies further validate the contributions of each module, highlighting the significance of advanced feature extraction and fusion strategies in enhancing emotion recognition performance. Our code are available at https://github.com/liangyubuaa/Milmer.

Figures

Figures reproduced from arXiv: 2502.00547 by the authors.

Figure 1
Figure 1. The overview of our framework, consisting of a EEG preprocessing module, a [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. The image feature dimensionality reduction module in this work. A cross [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Confusion matrices of subject-dependent and subject-independent experiments. [PITH_FULL_IMAGE:figures/full_fig_p021_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. WorldWeaver: Generating Long-Horizon Video Worlds via Rich Perception

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    WorldWeaver reduces temporal drift in long-horizon video generation by jointly modeling RGB and depth perceptual conditions with segmented noise scheduling.

  2. Foundation Models for Cross-Domain EEG Analysis Application: A Survey

    cs.HC 2025-08 conditional novelty 4.0 of 10

    A survey that organizes EEG foundation-model research into five output-modality categories: native EEG, text, vision, audio, and multimodal fusion, with a claim to be the first such comprehensive taxonomy.

  3. Evaluating the Impact of AI-Powered Audiovisual Personalization on Learner Emotion, Focus, and Learning Outcomes

    cs.AI 2025-05 unverdicted novelty 4.0 of 10

    The paper is a design and evaluation plan for an AI-generated personalized audiovisual study environment, with no empirical results reported.

Reference graph

Works this paper leans on

48 extracted references · 43 canonical work pages · cited by 3 Pith papers

  1. [1]

    Deap: A database for emotion anal- ysis; using physiological signals,

    S. Koelstra, C. Muhl, M. Soleymani, J.-S. Lee, A. Yazdani, T. Ebrahimi, T. Pun, A. Nijholt, and I. Patras, “Deap: A database for emotion anal- ysis; using physiological signals,” IEEE transactions on affective com- puting, vol. 3, no. 1, pp. 18–31, 2011

    work page 2011

  2. [2]

    Combining facial expressions and electroencephalography to enhance emotion recognition,

    Y. Huang, J. Yang, S. Liu, and J. Pan, “Combining facial expressions and electroencephalography to enhance emotion recognition,” Future Internet, vol. 11, no. 5, p. 105, 2019

    work page 2019

  3. [3]

    Facial emotion recognition: State of the art performance on fer2013,

    Y. Khaireddin and Z. Chen, “Facial emotion recognition: State of the art performance on fer2013,” arXiv preprint arXiv:2105.03588 , 2021

    arXiv 2021

  4. [4]

    C-gcn: Correlation based graph convolutional network for audio-video emotion recognition,

    W. Nie, M. Ren, J. Nie, and S. Zhao, “C-gcn: Correlation based graph convolutional network for audio-video emotion recognition,”IEEE Transactions on Multimedia, vol. 23, pp. 3793–3804, 2020

    work page 2020

  5. [5]

    Husformer: A multi-modal transformer for multi-modal human state recognition,

    R. Wang, W. Jo, D. Zhao, W. Wang, A. Gupte, B. Yang, G. Chen, and B.-C. Min, “Husformer: A multi-modal transformer for multi-modal human state recognition,” IEEE Transactions on Cognitive and Devel- opmental Systems , 2024

    work page 2024

  6. [6]

    Fetcheeg: a hybrid approach combining feature extraction and temporal-channel joint attention for eeg-based emotion classification,

    Y. Liang, C. Zhang, S. An, Z. Wang, K. Shi, T. Peng, Y. Ma, X. Xie, J. He, and K. Zheng, “Fetcheeg: a hybrid approach combining feature extraction and temporal-channel joint attention for eeg-based emotion classification,” Journal of Neural Engineering , vol. 21, no. 3, p. 036011, 2024

    work page 2024

  7. [7]

    A review of emotion recognition using physiological signals,

    L. Shu, J. Xie, M. Yang, Z. Li, Z. Li, D. Liao, X. Xu, and X. Yang, “A review of emotion recognition using physiological signals,” Sensors, vol. 18, no. 7, p. 2074, 2018

    work page 2018

  8. [8]

    A fusion framework for confusion analysis in learning based on eeg signals,

    C. Zhang, J. He, Y. Liang, Z. Wang, and X. Xie, “A fusion framework for confusion analysis in learning based on eeg signals,” Applied Sciences, vol. 13, no. 23, p. 12832, 2023. 27

    work page 2023

Show all 48 references
  1. [9]

    Emotion recognition us- ing multi-modal data and machine learning techniques: A tutorial and review,

    J. Zhang, Z. Yin, P. Chen, and S. Nichele, “Emotion recognition us- ing multi-modal data and machine learning techniques: A tutorial and review,” Information Fusion, vol. 59, pp. 103–126, 2020

    2020

  2. [10]

    Fusion of facial expressions and eeg for multimodal emotion recognition,

    Y. Huang, J. Yang, P. Liao, and J. Pan, “Fusion of facial expressions and eeg for multimodal emotion recognition,” Computational intelli- gence and neuroscience, vol. 2017, no. 1, p. 2107451, 2017

    2017

  3. [11]

    Deep learning model for simultaneous recognition of quantitative and qualitative emo- tion using visual and bio-sensing data,

    I. Hosseini, M. Z. Hossain, Y. Zhang, and S. Rahman, “Deep learning model for simultaneous recognition of quantitative and qualitative emo- tion using visual and bio-sensing data,” Computer Vision and Image Understanding, vol. 248, p. 104121, 2024

    2024

  4. [12]

    A review of multimodal emotion recognition from datasets, preprocessing, features, and fusion methods,

    B. Pan, K. Hirota, Z. Jia, and Y. Dai, “A review of multimodal emotion recognition from datasets, preprocessing, features, and fusion methods,” Neurocomputing, p. 126866, 2023

    2023

  5. [13]

    Vilt: Vision-and-language transformer without convolution or region supervision,

    W. Kim, B. Son, and I. Kim, “Vilt: Vision-and-language transformer without convolution or region supervision,” in International conference on machine learning , pp. 5583–5594, PMLR, 2021

    2021

  6. [14]

    Swin transformer: Hierarchical vision transformer using shifted win- dows,

    Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted win- dows,” in Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012–10022, 2021

    2021

  7. [15]

    Deep learning with convolutional neural networks for eeg decoding and visualization,

    R. T. Schirrmeister, J. T. Springenberg, L. D. J. Fiederer, M. Glasstet- ter, K. Eggensperger, M. Tangermann, F. Hutter, W. Burgard, and T. Ball, “Deep learning with convolutional neural networks for eeg decoding and visualization,” Human brain mapping , vol. 38, no. 11, pp. 5...

    2017

  8. [16]

    Identifying stable patterns over time for emotion recognition from eeg,

    W.-L. Zheng, J.-Y. Zhu, and B.-L. Lu, “Identifying stable patterns over time for emotion recognition from eeg,” IEEE transactions on affective computing, vol. 10, no. 3, pp. 417–429, 2017

    2017

  9. [17]

    Transformers for eeg- based emotion recognition: A hierarchical spatial information learning model,

    Z. Wang, Y. Wang, C. Hu, Z. Yin, and Y. Song, “Transformers for eeg- based emotion recognition: A hierarchical spatial information learning model,” IEEE Sensors Journal , vol. 22, no. 5, pp. 4359–4368, 2022. 28

    2022

  10. [18]

    Eeg-based emotion recognition us- ing regularized graph neural networks,

    P. Zhong, D. Wang, and C. Miao, “Eeg-based emotion recognition us- ing regularized graph neural networks,” IEEE Transactions on Affective Computing, vol. 13, no. 3, pp. 1290–1301, 2020

    2020

  11. [19]

    Emotion recognition in eeg signals using deep learning methods: A review,

    M. Jafari, A. Shoeibi, M. Khodatars, S. Bagherzadeh, A. Shalbaf, D. L. Garc ´ ıa, J. M. Gorriz, and U. R. Acharya, “Emotion recognition in eeg signals using deep learning methods: A review,” Computers in Biology and Medicine, p. 107450, 2023

    2023

  12. [20]

    A deep learning perspective on the origin of facial expressions,

    R. Breuer and R. Kimmel, “A deep learning perspective on the origin of facial expressions,” arXiv preprint arXiv:1705.01842 , 2017

    2017 arXiv

  13. [21]

    Video-based emotion recognition using cnn-rnn and c3d hybrid networks,

    Y. Fan, X. Lu, D. Li, and Y. Liu, “Video-based emotion recognition using cnn-rnn and c3d hybrid networks,” inProceedings of the 18th ACM international conference on multimodal interaction , pp. 445–450, 2016

    2016

  14. [22]

    End-to-end continuous emotion recognition from video using 3d convlstm networks,

    J. Huang, Y. Li, J. Tao, Z. Lian, and J. Yi, “End-to-end continuous emotion recognition from video using 3d convlstm networks,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), pp. 6837–6841, IEEE, 2018

    2018

  15. [23]

    Former-dfer: Dynamic facial expression recognition transformer,

    Z. Zhao and Q. Liu, “Former-dfer: Dynamic facial expression recognition transformer,” in Proceedings of the 29th ACM International Conference on Multimedia, pp. 1553–1561, 2021

    2021

  16. [24]

    Facial expression recog- nition with grid-wise attention and visual transformer,

    Q. Huang, C. Huang, X. Wang, and F. Jiang, “Facial expression recog- nition with grid-wise attention and visual transformer,” Information Sciences, vol. 580, pp. 35–54, 2021

    2021

  17. [25]

    A survey on facial emotion recognition techniques: A state-of-the-art literature review,

    F. Z. Canal, T. R. M¨ uller, J. C. Matias, G. G. Scotton, A. R. de Sa Ju- nior, E. Pozzebon, and A. C. Sobieranski, “A survey on facial emotion recognition techniques: A state-of-the-art literature review,” Informa- tion Sciences, vol. 582, pp. 593–617, 2022

    2022

  18. [26]

    A multi- modal emotion recognition method based on facial expressions and elec- troencephalography,

    Y. Tan, Z. Sun, F. Duan, J. Sol´ e-Casals, and C. F. Caiafa, “A multi- modal emotion recognition method based on facial expressions and elec- troencephalography,” Biomedical Signal Processing and Control, vol. 70, p. 103029, 2021. 29

    2021

  19. [27]

    A bimodal emotion recognition approach through the fusion of electroencephalography and facial sequences,

    F. Muhammad, M. Hussain, and H. Aboalsamh, “A bimodal emotion recognition approach through the fusion of electroencephalography and facial sequences,” Diagnostics, vol. 13, no. 5, p. 977, 2023

    2023

  20. [28]

    Efficient low-rank multimodal fusion with modality- specific factors,

    Z. Liu, Y. Shen, V. B. Lakshminarasimhan, P. P. Liang, A. Zadeh, and L.-P. Morency, “Efficient low-rank multimodal fusion with modality- specific factors,” arXiv preprint arXiv:1806.00064 , 2018

    2018 arXiv

  21. [29]

    Utilizing deep learning towards multi-modal bio-sensing and vision-based affective computing,

    T.-P. Jung, T. J. Sejnowski, et al. , “Utilizing deep learning towards multi-modal bio-sensing and vision-based affective computing,” IEEE Transactions on Affective Computing , vol. 13, no. 1, pp. 96–107, 2019

    2019

  22. [30]

    Attention-based deep multi- ple instance learning,

    M. Ilse, J. Tomczak, and M. Welling, “Attention-based deep multi- ple instance learning,” in International conference on machine learning, pp. 2127–2136, PMLR, 2018

    2018

  23. [31]

    Multi-scale blocks based image emotion classification using multiple instance learning,

    T. Rao, M. Xu, H. Liu, J. Wang, and I. Burnett, “Multi-scale blocks based image emotion classification using multiple instance learning,” in 2016 IEEE International Conference on Image Processing (ICIP) , pp. 634–638, IEEE, 2016

    2016

  24. [32]

    Multiple instance learning for emotion recognition using physiologi- cal signals,

    L. Romeo, A. Cavallo, L. Pepa, N. Bianchi-Berthouze, and M. Pontil, “Multiple instance learning for emotion recognition using physiologi- cal signals,” IEEE Transactions on affective computing , vol. 13, no. 1, pp. 389–407, 2019

    2019

  25. [33]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,”Advances in Neural Information Processing Systems, 2017

    2017

  26. [34]

    Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language mod- els,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language mod- els,” in International conference on machine learning , pp. 19730–19742, PMLR, 2023

    2023

  27. [35]

    Meg and eeg data analysis with mne-python,

    A. Gramfort, M. Luessi, E. Larson, D. A. Engemann, D. Strohmeier, C. Brodbeck, R. Goj, M. Jas, T. Brooks, L. Parkkonen, et al. , “Meg and eeg data analysis with mne-python,” Frontiers in Neuroinformatics, vol. 7, p. 267, 2013

    2013

  28. [36]

    Sgdr: Stochastic gradient descent with warm restarts,

    I. Loshchilov and F. Hutter, “Sgdr: Stochastic gradient descent with warm restarts,” arXiv preprint arXiv:1608.03983 , 2016. 30

    2016 arXiv

  29. [37]

    Blazeface: Sub-millisecond neural face detection on mobile gpus,

    V. Bazarevsky, Y. Kartynnik, A. Vakunov, K. Raveendran, and M. Grundmann, “Blazeface: Sub-millisecond neural face detection on mobile gpus,” arXiv preprint arXiv:1907.05047 , 2019

    1907 arXiv

  30. [38]

    Cross-subject emotion recognition using flexible analytic wavelet transform from eeg signals,

    V. Gupta, M. D. Chopda, and R. B. Pachori, “Cross-subject emotion recognition using flexible analytic wavelet transform from eeg signals,” IEEE Sensors Journal , vol. 19, no. 6, pp. 2266–2274, 2018

    2018

  31. [39]

    Electroencephalography based fusion two-dimensional (2d)-convolution neural networks (cnn) model for emotion recognition system,

    Y.-H. Kwon, S.-B. Shin, and S.-D. Kim, “Electroencephalography based fusion two-dimensional (2d)-convolution neural networks (cnn) model for emotion recognition system,” Sensors, vol. 18, no. 5, p. 1383, 2018

    2018

  32. [40]

    Eeg-based emotion recogni- tion using genetic algorithm optimized multi-layer perceptron,

    S. Marjit, U. Talukdar, and S. M. Hazarika, “Eeg-based emotion recogni- tion using genetic algorithm optimized multi-layer perceptron,” in 2021 International Symposium of Asian Control Association on Intelligent Robotics and Industrial Automation (IRIA) , pp. 304–309, IEEE, 2021

    2021

  33. [41]

    Cross-subject multi- modal emotion recognition based on hybrid fusion,

    Y. Cimtay, E. Ekmekcioglu, and S. Caglar-Ozhan, “Cross-subject multi- modal emotion recognition based on hybrid fusion,” IEEE Access, vol. 8, pp. 168865–168878, 2020

    2020

  34. [42]

    Emotion recognition using eeg signals and audiovisual features with contrastive learning,

    J.-H. Lee, J.-Y. Kim, and H.-G. Kim, “Emotion recognition using eeg signals and audiovisual features with contrastive learning,” Bioengineer- ing, vol. 11, no. 10, p. 997, 2024

    2024

  35. [43]

    Recogniz- ing, fast and slow: Complex emotion recognition with facial expression detection and remote physiological measurement,

    Y.-C. Wu, L.-W. Chiu, C.-C. Lai, B.-F. Wu, and S. S. Lin, “Recogniz- ing, fast and slow: Complex emotion recognition with facial expression detection and remote physiological measurement,” IEEE Transactions on Affective Computing , vol. 14, no. 4, pp. 3177–3190, 2023

    2023

  36. [44]

    Eeg-based emotion recognition using 3d convolutional neural net- works,

    E. S. Salama, R. A. El-Khoribi, M. E. Shoman, and M. A. W. Shal- aby, “Eeg-based emotion recognition using 3d convolutional neural net- works,” International Journal of Advanced Computer Science and Ap- plications, vol. 9, no. 8, 2018

    2018

  37. [45]

    Design and analysis of a closed- loop emotion regulation system based on multimodal affective comput- ing and emotional markov chain,

    X. Wang, C.-Z. Li, Z. Sun, and Y. Xu, “Design and analysis of a closed- loop emotion regulation system based on multimodal affective comput- ing and emotional markov chain,” IEEE Transactions on Systems, Man, and Cybernetics: Systems , 2025. 31

    2025

  38. [46]

    Expression eeg multimodal emotion recogni- tion method based on the bidirectional lstm and attention mechanism,

    Y. Zhao and D. Chen, “Expression eeg multimodal emotion recogni- tion method based on the bidirectional lstm and attention mechanism,” Computational and Mathematical Methods in Medicine , vol. 2021, no. 1, p. 9967592, 2021

    2021

  39. [47]

    Multi-modal emotion identification fusing facial ex- pression and eeg,

    Y. Wu and J. Li, “Multi-modal emotion identification fusing facial ex- pression and eeg,” Multimedia Tools and Applications , vol. 82, no. 7, pp. 10901–10919, 2023

    2023

  40. [48]

    Multimodal emotion recog- nition from eeg signals and facial expressions,

    S. Wang, J. Qu, Y. Zhang, and Y. Zhang, “Multimodal emotion recog- nition from eeg signals and facial expressions,” IEEE Access, vol. 11, pp. 33061–33068, 2023. 32

    2023

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.