Pith. sign in

REVIEW 5 major objections 5 minor 69 references

SDR-GNN: Spectral Domain Reconstruction Graph Neural Network for Incomplete Multimodal Learning in Conversational Emotion Recognition

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SDR-GNN claims that a spectral-domain graph network, by keeping high-frequency signals alive during aggregation, can reconstruct missing modalities and sustain conversational emotion recognition performance up to 70% missing rates.

desk verdict A plausible GNN combination for incomplete multimodal ERC, but the SOTA claim is undermined by an internal table conflict and by the paper's own MOSI numbers. read the letter →

arxiv 2411.19822 v1 pith:OVEWIBMB submitted 2024-11-29 cs.CL

Fangze Fu , Wei Ai , Fan Yang , Yuntao Shou , Tao Meng , Keqin Li This is my paper · ORCID
classification cs.CL
keywords IncompletemultimodallearningConversationalemotionrecognitionfusionSpectraldomainreconstructionGraphneuralnetworkHypergraphHigh-frequencyinformationModalityimputation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a graph neural network operating in the spectral domain can recover missing audio, visual, and text features in conversation, so that emotion recognition stays accurate even when most modality data is absent. It argues that previous graph methods lose high-frequency signals—the sharp emotional shifts that distinguish anger from neutral, for example—because message passing smooths them away. SDR-GNN builds a speaker-and-context interaction graph, enriches it with a weighted hypergraph that captures multi-node relationships, and adds a self-gating mechanism that keeps both low- and high-frequency information during aggregation. It then reconstructs missing modalities with a linear layer and fuses everything through multi-head attention. Across IEMOCAP, CMU-MOSI, and CMU-MOSEI, the authors report consistent gains over comparison methods, with the largest advantages at high missing rates.

What carries the argument

The central mechanism is a graph built from a sliding context window over utterances, with two edge types—speaker identity edges and context-position edges—that feed relation-aware graph convolutions. On top of this sits a weighted hypergraph with edge-dependent node weights, which lets one hyperedge connect multiple utterances and capture higher-order dependencies. The frequency-aware module is a self-gating aggregation: for each edge, a gating score computed from the concatenated node pair determines whether low-frequency (similarity) or high-frequency (difference) signals dominate, using tanh to scale the gate between -1 and 1. This gating drives the reconstruction layer, which predicts missing modality features from the learned representations, and the multi-head attention layer that refines the reconstructed data.

What would settle it

Run SDR-GNN on IEMOCAP with a test set where one speaker's audio is entirely removed while text and video stay complete, matching the same overall missing proportion as a uniform-random condition, and compare WAF1. If performance falls to the level of a no-audio unimodal baseline, the claim that multi-frequency aggregation alone drives robustness fails; a second check is to train with some utterances having all three modalities masked and observe whether the reconstruction layer can produce any signal at all.

Watch

Extended reading notes

Core claim

The paper's central claim is that SDR-GNN outperforms current state-of-the-art methods for incomplete multimodal emotion recognition in conversations, on all three benchmark datasets and across missing rates from 0.0 to 0.7. The reported gains are 0.77% to 8.6% in weighted average F1 over baselines, and the model degrades less than others as the missing rate rises: on IEMOCAP four-class, its WAF1 drops by only 5.17% from missing rate 0.0 to 0.7, while comparison methods drop between 6.98% and 37.70%. The paper attributes this robustness to two design choices: a weighted hypergraph that models higher-order utterance dependencies beyond binary edges, and a frequency-aware aggregation that preserves high-frequency information that ordinary graph convolutions over-smooth away. The reconstruction layer then uses the multi-frequency-enriched features to impute missing modalities, and a multi-head attention layer optimizes the recovered data before classification.

Load-bearing premise

Every utterance keeps at least one of its three modalities, and missing modalities are chosen randomly with uniform probability; if an utterance loses all modalities, or missingness is patterned (like one speaker's audio always gone), the reconstruction layer has nothing to work from and the claimed robustness is not guaranteed.

Editorial extensions

If this is right

  • At missing rate 0.7, SDR-GNN declines only 5.17% on IEMOCAP four-class, while baselines drop between 6.98% and 37.70%, so the model's robustness gap widens precisely where data is most incomplete.
  • On fully complete data (missing rate 0.0), SDR-GNN still improves WAF1 by 0.7% to 2.7% over baselines, meaning the multi-frequency architecture helps even without any missing modalities.
  • Reconstruction quality, measured by MSE between imputed and true features, stays lowest for SDR-GNN and grows more slowly as the missing rate increases, indicating that the frequency-aware features genuinely improve imputation.
  • Discarding incomplete utterances (the lower-bound setting) consistently hurts performance compared with using them, so the incomplete data itself carries signal that SDR-GNN exploits.
  • The case study shows SDR-GNN and MMIN preserve a high-frequency emotional transition (a shift from neutral to anger) that other context-based models miss, which supports the claim that high-frequency preservation matters for emotion recognition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The uniform-random masking assumption likely overstates real-world robustness; correlated missing patterns, such as one speaker's audio being entirely absent, should be tested to see whether the reconstruction layer can still interpolate from a missing-stream context.
  • The self-gating mechanism might transfer to other conversational tasks where abrupt state changes matter, such as sarcasm detection or dialogue-act shift detection, because it explicitly preserves node-level differences that standard smoothing erases.
  • Since the reconstruction loss weight $e=0.5$ was found optimal, an adaptive weighting scheme that increases reconstruction pressure as the missing rate grows could extend the method beyond 0.7, although the at-least-one-modality constraint would still bind.
  • The paper's attribution of gains to high-frequency preservation could be tested directly by flipping the sign of the gating scores (making the model favor smoothing), which should sharply reduce performance at high missing rates if the mechanism is load-bearing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes SDR-GNN, a graph neural network for incomplete multimodal emotion recognition in conversations. SDR-GNN builds speaker and context interaction graphs over utterances, adds a hypergraph and a learned gating mechanism described as frequency-aware, reconstructs missing modality features, and classifies emotions. It is evaluated on IEMOCAP, CMU-MOSI, and CMU-MOSEI at missing rates from 0.0 to 0.7, reporting weighted F1 and accuracy, and the abstract claims that the method "outperforms current state-of-the-art methods."

Significance. If the reported results were reliable, SDR-GNN would be a meaningful contribution to incomplete multimodal learning in conversational emotion recognition. The paper has notable strengths: the authors provide public code, a detailed ablation study (Table 4), explicit parameter tuning (Section 5.7), a reconstruction-error comparison (Figure 5), and a case study (Figure 8). However, the central empirical claim is not currently supported: the paper's own tables contain a direct numerical contradiction, and several baselines actually beat the proposed method on CMU-MOSI. The spectral/frequency interpretation of the core mechanism is also not demonstrated. These issues are load-bearing for the claimed contribution.

major comments (5)
  1. [Table 2 vs Table 5] There is a direct internal inconsistency in the reported results. For IEMOCAP four-class at missing rate 0.6, Table 2 gives SDR-GNN WAF1=75.03, GCNet=71.88, and CPM-Net=47.38, while Table 5 gives SDR-GNN=81.13, GCNet=78.87, and CPM-Net=68.68 for the same dataset, same missing rate, and same metric. At least one set of numbers is wrong, or the experimental configurations differ in a way the paper does not disclose. Since the abstract's SOTA claim rests on Tables 2 and 3, this contradiction undermines the central empirical result and must be resolved before the paper can be evaluated.
  2. [Table 3, CMU-MOSI] In the paper's own comparison, DiCMoR outperforms SDR-GNN on CMU-MOSI at missing rates 0.2 (82.0/82.1 vs. 81.9/81.9), 0.5 (76.4/76.7 vs. 76.1/76.2), and 0.6 (73.0/73.3 vs. 72.2/72.2). This contradicts the abstract's claim that SDR-GNN outperforms current state-of-the-art methods. The authors should either qualify the claim (e.g., "outperforms on IEMOCAP and CMU-MOSEI") or provide statistical evidence, such as significance tests over repeated runs, that supports an overall superiority claim.
  3. [Tables 2-3, baselines] The baseline numbers are largely borrowed from previous papers: the footnote to Table 2 states "results come from [22]" and Table 3 states "results come from [60]." These numbers were obtained under different experimental protocols, including different masking implementations, random seeds, and possibly different feature preprocessing or data splits. The paper does not rerun any baseline under its own masking protocol, and it reports no standard deviation or confidence interval despite stating in Section 4.2 that performance is averaged over ten trials. Without a shared protocol and variance reporting, the comparative claims in Tables 2 and 3 are not quantitatively meaningful.
  4. [Section 3.2, Eqs. (8)-(9)] The paper's central novelty is described as "Spectral Domain Reconstruction" and "multi-frequency aggregation," but Equations (8) and (9) implement a learned gating mechanism: a tanh-scaled linear function of concatenated node features. No graph Laplacian, no spectral decomposition, and no frequency-domain filtering appears in the model. The statement that "These signals are derived through the spectral decomposition of the graph Laplacian" (Section 3.2) is not matched by any mathematical or experimental analysis of the learned filters. The authors should either provide evidence that the gating mechanism actually separates high- and low-frequency graph signal components, or reframe the contribution as a gated aggregation method without the spectral claim.
  5. [Section 3.1 and Section 4.2] The evaluation is restricted to a specific missingness model: randomly masking individual utterance modalities while ensuring at least one modality remains per utterance. Section 3.1 states this assumption explicitly, and Section 4.2 confirms that rho ranges only to 0.7, the approximate maximum under the at-least-one-modality constraint. Real conversations can contain utterances with all modalities missing (e.g., sensor failure) or structured missing patterns (e.g., an entire speaker's audio channel absent). The paper does not discuss or test these cases, so the claimed robustness of the reconstruction layer (Eq. 11) is bounded by this assumption. The authors should state this limitation in the abstract and conclusion and, ideally, evaluate a setting with all-modality-missing utterances.
minor comments (5)
  1. [Section 5.5] The text says "in Fig. 3, we compare the performance" but Figure 3 shows confusion matrices; the correct cross-reference appears to be Figure 4, which shows the lower-bound comparison.
  2. [Section 5.7] The text refers to "Fig. 6" for the reconstruction-loss weight e and "Fig. 7" for the number of layers l, but the figure captions are reversed: Figure 6 is captioned "Parameter tuning with various number of hypergraph" and Figure 7 is captioned "various weight of reconstruction loss function." The cross-references must be fixed.
  3. [Section 5.2] There are typos in the ablation discussion: "spearker" should be "speaker," and "Codeceases" appears to be a typo for "decreases." Also, Section 5.2 defines SDR-GNNw/o Fre as replacing the frequency-aware layer with DialogueGCN, but the text later says "we replace the frequency-aware graph learning with a graph convolution network from DialogueGCN," which is clear after the typo fix.
  4. [Section 3.3, Eq. (16)] The reconstruction loss computes the MSE between the reconstructed and original features, but when a modality is missing, the original is not observed in the test setting. The paper should clarify that this loss is computed only during training where the missingness is artificially simulated, since otherwise Eq. (16) would require access to the ground-truth missing features.
  5. [Section 3.2, Eq. (3)] The incidence matrix in Eq. (3) is denoted with a hat as \hat{H} and described as a weighted incidence matrix, but Eq. (6) later uses both H and \hat{H}; the relationship between the unweighted and weighted matrices should be defined explicitly to avoid ambiguity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: SDR-GNN's benchmark claims are empirical outputs of a trained model, not consequences of its assumptions; the multi-frequency framing is interpretive and the Table 2/Table 5 discrepancy is a consistency concern, not a circularity.

full rationale

SDR-GNN makes no first-principles prediction that could be equivalent to its inputs by construction. The headline claim is a set of empirical WAF1/ACC numbers obtained by training on standard splits (IEMOCAP, CMU-MOSI, CMU-MOSEI) under a fixed missing-rate protocol; hyperparameters are chosen on validation data and the losses (Eqs. 15-17) are ordinary cross-entropy plus reconstruction error, neither of which encodes the reported test scores. No parameter is fitted to the test-set labels, and no uniqueness theorem or load-bearing prior result is imported from the authors' other papers; self-citations appear only in related-work enumeration. The 'multi-frequency' narrative around Eqs. 8-9 is a post hoc interpretation of a learned tanh gate rather than an implemented spectral decomposition, so it is a naming/attribution overclaim, not a circular reduction. In addition, Table 2 and Table 5 report incompatible values for the same IEMOCAP four-class 0.6 condition (Ours 75.03 vs 81.13; GCNet 71.88 vs 78.87), which undermines the empirical SOTA claim, but internal inconsistency is a correctness risk, not circularity. Therefore no circular step is established and the derivation chain is self-contained as reported.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The model relies on explicit assumptions about modality availability and random masking, plus a post hoc interpretation of a gating mechanism as frequency separation. No new physical or conceptual entities are introduced beyond the architecture itself.

free parameters (4)
  • Window size w = 1-4 (selected per dataset on validation, not reported)
    Sliding window size for context/speaker graphs, tuned in Table 6.
  • Hidden dimension h = 100-250 (selected per dataset, not reported)
    Latent representation size, tuned in Table 6.
  • Reconstruction loss weight e = 0.5
    Tuned on IEMOCAP, Fig. 7.
  • Number of hypergraph layers l = not specified for final model
    Tuned on IEMOCAP, Fig. 6.
assumptions (4)
  • domain assumption At least one modality remains available for every utterance
    Stated in Section 3.1; the reconstruction layer needs input from the remaining modalities.
  • domain assumption Missing modalities are randomly masked with a uniform distribution governed by the missing rate r
    Section 4.2; real-world missing patterns may be structured or correlated.
  • ad hoc to paper Baseline results in Tables 2 and 3, taken from earlier papers, are directly comparable despite different masking protocols
    The paper does not rerun baselines or verify that missing masks match.
  • ad hoc to paper The self-gating mechanism in Eqs. 8-9 corresponds to a separation of high- and low-frequency graph signals
    No spectral decomposition is performed; the claim is asserted in Section 3.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SDR-GNN: Spectral Domain Reconstruction Graph Neural Network for Incomplete Multimodal Learning in Conversational Emotion Recognition." pith.science (2026). https://pith.science/paper/OVEWIBMB

@misc{pith2026241119822,
  author       = {Pith},
  title        = {Pith review of: SDR-GNN: Spectral Domain Reconstruction Graph Neural Network for Incomplete Multimodal Learning in Conversational Emotion Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OVEWIBMB}},
  note         = {Machine review of arXiv:2411.19822}
}
read the original abstract

Multimodal Emotion Recognition in Conversations (MERC) aims to classify utterance emotions using textual, auditory, and visual modal features. Most existing MERC methods assume each utterance has complete modalities, overlooking the common issue of incomplete modalities in real-world scenarios. Recently, graph neural networks (GNNs) have achieved notable results in Incomplete Multimodal Emotion Recognition in Conversations (IMERC). However, traditional GNNs focus on binary relationships between nodes, limiting their ability to capture more complex, higher-order information. Moreover, repeated message passing can cause over-smoothing, reducing their capacity to preserve essential high-frequency details. To address these issues, we propose a Spectral Domain Reconstruction Graph Neural Network (SDR-GNN) for incomplete multimodal learning in conversational emotion recognition. SDR-GNN constructs an utterance semantic interaction graph using a sliding window based on both speaker and context relationships to model emotional dependencies. To capture higher-order and high-frequency information, SDR-GNN utilizes weighted relationship aggregation, ensuring consistent semantic feature extraction across utterances. Additionally, it performs multi-frequency aggregation in the spectral domain, enabling efficient recovery of incomplete modalities by extracting both high- and low-frequency information. Finally, multi-head attention is applied to fuse and optimize features for emotion recognition. Extensive experiments on various real-world datasets demonstrate that our approach is effective in incomplete multimodal learning and outperforms current state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2411.19822 by the authors.

Figure 1
Figure 1. A toy example of complete multimodal features and incomplete multimodal features in conversation. Missing modalities pose a considerable challenge to capturing intra￾and inter-modal semantic dependencies. utterance contains three modalities. The conversation on the right side illustrates the condition when modalities are incomplete. The problem of incomplete modalities poses significant challenges for MERC tasks. To… view at source ↗
Figure 2
Figure 2. The overall structure of the framework. First, we encode features of the utterance using a Bi-GRU to obtain the contextual embedding of each node. Then, we apply the SDR-GNN to capture features, jointly considering higher-order and multi-frequency information. Finally, we reconstruct the incomplete features and classify the emotion labels. In this paper, we assume at least one modality-complete data is available for… view at source ↗
Figure 3
Figure 3. Confusion matrices of the test set on IEMOCAP at varying missing rates. The matrices present the true labels along its rows and the predicted labels across its columns. our SDR-GNN can effectively recognize conversations with high missing rates. However, as the missing rate increases, we notice that conversations truly labeled as ‘happy’ are more likely to be misclassified as ‘angry’. We attribute this to the possib… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Classification performance comparison between SDR-GNN and Lower bound under different missing rates [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Reconstruction performance comparison between SDR-GNN and other methods under different missing rates. Lower MSE indicates better imputation performance. the performance of conversational learning models. By ef￾fectively incorporating incomplete information, SDR-GNN de…
Figure 6
Figure 6. Figure 6: Parameter tuning with various number of hypergraph from IEMOCAP(Four) [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Parameter tuning with various weight of reconstruc￾tion loss function from IEMOCAP (Four-class). more irrelevant information. This will increase the difficulty of model learning. Similarly, an increase in the hidden layer dimension ℎ generally results in improved perfo…
Figure 8
Figure 8. Figure 8: Prediction results on incomplete conversational data from IEMOCAP (Four-class). performance first rises and then declines, with the best performance observed around 𝑒 = 0.5. We believe that the reconstruction task and the classification task should have similar weights…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 53 canonical work pages

  1. [22]

    Gcnet:Graphcom- pletion network for incomplete multimodal learning in conversation

    Lian,Z.,Chen,L.,Sun,L.,Liu,B.,Tao,J.,2023. Gcnet:Graphcom- pletion network for incomplete multimodal learning in conversation. IEEE Transactions on pattern analysis and machine intelligence

    work page 2023

  2. [60]

    Distribution-consistent modal recoveringforincompletemultimodallearning,in:Proceedingsofthe IEEE/CVFInternationalConferenceonComputerVision(ICCV),pp

    Wang, Y., Cui, Z., Li, Y., 2023. Distribution-consistent modal recoveringforincompletemultimodallearning,in:Proceedingsofthe IEEE/CVFInternationalConferenceonComputerVision(ICCV),pp. 22025–22034

    work page 2023

  3. [1]

    Mcsff: Multi-modal consistency and specificity fusion framework for entity alignment

    Ai,W.,Deng,W.,Chen,H.,Du,J.,Meng,T.,Shou,Y.,2024a. Mcsff: Multi-modal consistency and specificity fusion framework for entity alignment. arXiv preprint arXiv:2410.14584

    arXiv

  4. [2]

    Seg: Seeds-enhanced iterative refinement graph neural network for entity alignment

    Ai,W.,Gao,Y.,Li,J.,Du,J.,Meng,T.,Shou,Y.,Li,K.,2024b. Seg: Seeds-enhanced iterative refinement graph neural network for entity alignment. arXiv preprint arXiv:2410.20733

    arXiv

  5. [3]

    Graph contrastive learning via cluster-refined negative sampling for semi-supervisedtextclassification

    Ai, W., Li, J., Wang, Z., Du, J., Meng, T., Shou, Y., Li, K., 2024c. Graph contrastive learning via cluster-refined negative sampling for semi-supervisedtextclassification. arXivpreprintarXiv:2410.18130

    arXiv

  6. [4]

    Contrastive multi-graph learning with neighbor hierarchi- cal sifting for semi-supervised text classification

    Ai, W., Li, J., Wang, Z., Wei, Y., Meng, T., Shou, Y., Lib, K., 2024d. Contrastive multi-graph learning with neighbor hierarchi- cal sifting for semi-supervised text classification. arXiv preprint arXiv:2411.16787

    arXiv

  7. [5]

    Der-gcn:Dialogandevent relation-aware graph convolutional neural network for multimodal dialog emotion recognition

    Ai,W.,Shou,Y.,Meng,T.,Li,K.,2024e. Der-gcn:Dialogandevent relation-aware graph convolutional neural network for multimodal dialog emotion recognition. IEEE Transactions on Neural Networks and Learning Systems

    work page

  8. [6]

    Der-gcn: Dialogue and event relation-aware graph convolutional neural net- work for multimodal dialogue emotion recognition

    Ai, W., Shou, Y., Meng, T., Yin, N., Li, K., 2023a. Der-gcn: Dialogue and event relation-aware graph convolutional neural net- work for multimodal dialogue emotion recognition. arXiv preprint arXiv:2312.10579

    arXiv

Show all 69 references
  1. [7]

    Edge- enhanced minimum-margin graph attention network for short text classification

    Ai, W., Wei, Y., Shao, H., Shou, Y., Meng, T., Li, K., 2024f. Edge- enhanced minimum-margin graph attention network for short text classification. Expert Systems with Applications 251, 124069

  2. [8]

    A two-stage multimodal emotion recognition model based on graph contrastivelearning,in:2023IEEE29thInternationalConferenceon Parallel and Distributed Systems (ICPADS), IEEE

    Ai, W., Zhang, F., Meng, T., Shou, Y., Shao, H., Li, K., 2023b. A two-stage multimodal emotion recognition model based on graph contrastivelearning,in:2023IEEE29thInternationalConferenceon Parallel and Distributed Systems (ICPADS), IEEE. pp. 397–404

  3. [9]

    Deepcanonical correlationanalysis,in:Internationalconferenceonmachinelearning, PMLR

    Andrew,G.,Arora,R.,Bilmes,J.,Livescu,K.,2013. Deepcanonical correlationanalysis,in:Internationalconferenceonmachinelearning, PMLR. pp. 1247–1255

    2013

  4. [10]

    Hypergraph convolution and hypergraph attention

    Bai, S., Zhang, F., Torr, P.H., 2021. Hypergraph convolution and hypergraph attention. Pattern Recognition 110, 107637

    2021

  5. [11]

    Greedy layer-wisetrainingofdeepnetworks,in:ProceedingsoftheAdvances in Neural Information Processing Systems, pp

    Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H., 2007. Greedy layer-wisetrainingofdeepnetworks,in:ProceedingsoftheAdvances in Neural Information Processing Systems, pp. 153–160

    2007

  6. [12]

    Beyond low-frequency information in graph convolutional networks, in: Proceedings of the AAAI conference on artificial intelligence, pp

    Bo, D., Wang, X., Shi, C., Shen, H., 2021. Beyond low-frequency information in graph convolutional networks, in: Proceedings of the AAAI conference on artificial intelligence, pp. 3950–3957

    2021

  7. [13]

    Iemocap: Interactive emotional dyadic motion capture database

    Busso, C., Bulut, M., Lee, C.C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J.N., Lee, S., Narayanan, S.S., 2008. Iemocap: Interactive emotional dyadic motion capture database. Language Resources and Evaluation 42, 335–359

    2008

  8. [14]

    Deep adversarial learningformulti-modalitymissingdatacompletion,in:Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp

    Cai, L., Wang, Z., Gao, H., Shen, D., Ji, S., 2018. Deep adversarial learningformulti-modalitymissingdatacompletion,in:Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 1158–1166

    2018

  9. [15]

    Semeval- 2019task3:Emocontextcontextualemotiondetectionintext,in:Pro- ceedings of the 13th international workshop on semantic evaluation, pp

    Chatterjee,A.,Narahari,K.N.,Joshi,M.,Agrawal,P.,2019. Semeval- 2019task3:Emocontextcontextualemotiondetectionintext,in:Pro- ceedings of the 13th international workshop on semantic evaluation, pp. 39–48

    2019

  10. [16]

    Chen, F., Shao, J., Zhu, S., Shen, H.T., 2023. Multivariate, multi-frequency and multimodal: Rethinking graph neural networks for emotion recognition in conversation, in: Proceedings of the IEEE/CVFConferenceonComputerVisionandPatternRecognition, pp. 10761–10770

    2023

  11. [17]

    M2fnet: Multi-modal fusion network for emotion recogni- tioninconversation,in:ProceedingsoftheIEEE/CVFConferenceon Computer Vision and Pattern Recognition, pp

    Chudasama, V., Kar, P., Gudmalwar, A., Shah, N., Wasnik, P., Onoe, N., 2022. M2fnet: Multi-modal fusion network for emotion recogni- tioninconversation,in:ProceedingsoftheIEEE/CVFConferenceon Computer Vision and Pattern Recognition, pp. 4652–4661

    2022

  12. [18]

    Fusing pairwise modalities for emotion recognition in conversations

    Fan, C., Lin, J., Mao, R., Cambria, E., 2024. Fusing pairwise modalities for emotion recognition in conversations. Information Fusion , 102306

    2024

  13. [19]

    Dialoguegcn: A graph convolutional neural network for emotion recognition in conversation, in: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp

    Ghosal, D., Majumder, N., Poria, S., Chhaya, N., Gelbukh, A., 2019. Dialoguegcn: A graph convolutional neural network for emotion recognition in conversation, in: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 154–164

    2019

  14. [20]

    Relations between two sets of variates, in: Breakthroughs in statistics

    Hotelling, H., 1992. Relations between two sets of variates, in: Breakthroughs in statistics. Springer, pp. 162–190

    1992

  15. [21]

    Hu, J., Liu, Y., Zhao, J., Jin, Q., 2021. Mmgcn: Multimodal fu- sion via deep graph convolution network for emotion recognition in conversation, in: Proceedings of the 59th Annual Meeting of the AssociationforComputationalLinguisticsandthe11thInternational Joint Conference on ...

    2021

  16. [23]

    Com- pleter:Incompletemulti-viewclusteringviacontrastiveprediction,in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Lin, Y., Gou, Y., Liu, Z., Li, B., Lv, J., Peng, X., 2021. Com- pleter:Incompletemulti-viewclusteringviacontrastiveprediction,in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11174–11183

    2021

  17. [24]

    An efficient approach for audio-visual emotion recognition with missing labels and missing modalities, in: 2021 IEEE International Conference on Multimedia and Expo (ICME), IEEE

    Ma, F., Huang, S.L., Zhang, L., 2021a. An efficient approach for audio-visual emotion recognition with missing labels and missing modalities, in: 2021 IEEE International Conference on Multimedia and Expo (ICME), IEEE. pp. 1–6

    2021

  18. [25]

    Maximum likelihood estimation for multimodal learning with missing modality

    Ma, F., Xu, X., Huang, S.L., Zhang, L., 2021b. Maximum likelihood estimation for multimodal learning with missing modality. arXiv preprint arXiv:2108.10513

    arXiv

  19. [26]

    Dialoguernn: An attentive rnn for emotion detection in conversations, in: Proceedings of the AAAI Conference on Artificial Intelligence, pp

    Majumder, N., Poria, S., Hazarika, D., Mihalcea, R., Gelbukh, A., Cambria, E., 2019. Dialoguernn: An attentive rnn for emotion detection in conversations, in: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 6818–6825

    2019

  20. [27]

    A multi-message passing framework based on heterogeneous graphs in conversational emotion recognition

    Meng, T., Shou, Y., Ai, W., Du, J., Liu, H., Li, K., 2024a. A multi-message passing framework based on heterogeneous graphs in conversational emotion recognition. Neurocomputing 569, 127109

  21. [28]

    Deep imbalanced learning for multimodal emotion recognition in conversations

    Meng, T., Shou, Y., Ai, W., Yin, N., Li, K., 2024b. Deep imbalanced learning for multimodal emotion recognition in conversations. IEEE Transactions on Artificial Intelligence

  22. [29]

    Revisiting multimodal emotion recognition in conversation from the perspective of graph spectrum

    Meng, T., Zhang, F., Shou, Y., Ai, W., Yin, N., Li, K., 2024c. Revisiting multimodal emotion recognition in conversation from the perspective of graph spectrum. arXiv preprint arXiv:2404.17862

    arXiv

  23. [30]

    IEEE/ACM Transactions on Audio, Speech, and Language Processing

    Meng,T.,Zhang,F.,Shou,Y.,Shao,H.,Ai,W.,Li,K.,2024d.Masked graph learning with recurrent alignment for multimodal emotion recognition in conversation. IEEE/ACM Transactions on Audio, Speech, and Language Processing . Fu et al.:Preprint submitted to Elsevier Page 15 of 16 Fangze...

  24. [31]

    I-gcn:Incremental graphconvolutionnetworkforconversationemotiondetection

    Nie,W.,Chang,R.,Ren,M.,Su,Y.,Liu,A.,2021. I-gcn:Incremental graphconvolutionnetworkforconversationemotiondetection. IEEE Transactions on Multimedia 24, 4471–4481

    2021

  25. [32]

    Parthasarathy, S., Sundaram, S., 2020. Training strategies to han- dle missing modalities for audio-visual expression recognition, in: Companion Publication of the 2020 International Conference on Multimodal Interaction, pp. 400–404

    2020

  26. [33]

    Found in translation: Learning robust joint representations by cyclic translationsbetweenmodalities,in:ProceedingsoftheAAAIconfer- ence on artificial intelligence, pp

    Pham, H., Liang, P.P., Manzini, T., Morency, L.P., Póczos, B., 2019. Found in translation: Learning robust joint representations by cyclic translationsbetweenmodalities,in:ProceedingsoftheAAAIconfer- ence on artificial intelligence, pp. 6892–6899

    2019

  27. [34]

    Context-dependent sentiment analysis in user- generated videos, in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp

    Poria, S., Cambria, E., Hazarika, D., Majumder, N., Zadeh, A., Morency, L.P., 2017. Context-dependent sentiment analysis in user- generated videos, in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 873–883

    2017

  28. [35]

    Modelingrelationaldatawithgraphconvolutional networks,in:EuropeanSemanticWebConference,Springer.pp.593– 607

    Schlichtkrull, M., Kipf, T.N., Bloem, P., Van Den Berg, R., Titov, I., Welling,M.,2018. Modelingrelationaldatawithgraphconvolutional networks,in:EuropeanSemanticWebConference,Springer.pp.593– 607

    2018

  29. [36]

    14752–14766

    Shi,T.,Huang,S.L.,2023.Multiemo:Anattention-basedcorrelation- aware multimodal fusion framework for emotion recognition in con- versations, in: Proceedings of the 61st Annual Meeting of the Asso- ciation for Computational Linguistics (Volume 1: Long Papers), pp. 14752–14766

    2023

  30. [37]

    Effi- cient long-distance latent relation-aware graph neural network for multi-modal emotion recognition in conversations

    Shou, Y., Ai, W., Du, J., Meng, T., Liu, H., 2024a. Effi- cient long-distance latent relation-aware graph neural network for multi-modal emotion recognition in conversations. arXiv preprint arXiv:2407.00119

    arXiv

  31. [38]

    Czl-ciae:Clip-drivenzero- shot learning for correcting inverse age estimation

    Shou,Y.,Ai,W.,Meng,T.,Li,K.,2023a. Czl-ciae:Clip-drivenzero- shot learning for correcting inverse age estimation. arXiv preprint arXiv:2312.01758

    arXiv

  32. [39]

    Graph informa- tion bottleneck for remote sensing segmentation

    Shou, Y., Ai, W., Meng, T., Yin, N., 2023b. Graph informa- tion bottleneck for remote sensing segmentation. arXiv preprint arXiv:2312.02545

  33. [40]

    Graphunet: Graph make strong encoders for remote sensing segmentation, in: 2023IEEE29thInternationalConferenceonParallelandDistributed Systems (ICPADS), IEEE

    Shou, Y., Ai, W., Meng, T., Zhang, F., Li, K., 2023c. Graphunet: Graph make strong encoders for remote sensing segmentation, in: 2023IEEE29thInternationalConferenceonParallelandDistributed Systems (ICPADS), IEEE. pp. 2734–2737

  34. [41]

    Maskedcontrastivegraph representation learning for age estimation

    Shou,Y.,Cao,X.,Liu,H.,Meng,D.,2025. Maskedcontrastivegraph representation learning for age estimation. Pattern Recognition 158, 110974

    2025

  35. [42]

    Spegcl: Self-supervised graph spectrum contrastive learning without positive samples

    Shou, Y., Cao, X., Meng, D., 2024b. Spegcl: Self-supervised graph spectrum contrastive learning without positive samples. arXiv preprint arXiv:2410.10365

    arXiv

  36. [43]

    Contrastive graph representation learning with adversarial cross-view reconstruction and information bottleneck

    Shou, Y., Lan, H., Cao, X., 2024c. Contrastive graph representation learning with adversarial cross-view reconstruction and information bottleneck. arXiv preprint arXiv:2408.00295

    arXiv

  37. [44]

    A low- rank matching attention based cross-modal feature fusion method for conversational emotion recognition

    Shou, Y., Liu, H., Cao, X., Meng, D., Dong, B., 2024d. A low- rank matching attention based cross-modal feature fusion method for conversational emotion recognition. IEEE Transactions on Affective Computing

  38. [45]

    Objectdetectioninmedicalimagesbasedonhierarchicaltransformer and mask mechanism

    Shou, Y., Meng, T., Ai, W., Xie, C., Liu, H., Wang, Y., 2022a. Objectdetectioninmedicalimagesbasedonhierarchicaltransformer and mask mechanism. Computational Intelligence and Neuroscience 2022, 5863782

    2022

  39. [46]

    Conversational emotion recognition studies based on graph convolutional neural networks and a dependent syntactic analysis

    Shou, Y., Meng, T., Ai, W., Yang, S., Li, K., 2022b. Conversational emotion recognition studies based on graph convolutional neural networks and a dependent syntactic analysis. Neurocomputing 501, 629–639

  40. [47]

    Adver- sarial representation with intra-modal and inter-modal graph con- trastive learning for multimodal emotion recognition

    Shou, Y., Meng, T., Ai, W., Yin, N., Li, K., 2023d. Adver- sarial representation with intra-modal and inter-modal graph con- trastive learning for multimodal emotion recognition. arXiv preprint arXiv:2312.16778

    arXiv

  41. [48]

    A comprehensive surveyonmulti-modalconversationalemotionrecognitionwithdeep learning

    Shou, Y., Meng, T., Ai, W., Yin, N., Li, K., 2023e. A comprehensive surveyonmulti-modalconversationalemotionrecognitionwithdeep learning. arXiv preprint arXiv:2312.05735

  42. [49]

    Adversarial alignment and graph fusion via information bottleneck for multimodal emotion recognition in conversations

    Shou, Y., Meng, T., Ai, W., Zhang, F., Yin, N., Li, K., 2024e. Adversarial alignment and graph fusion via information bottleneck for multimodal emotion recognition in conversations. Information Fusion 112, 102590

  43. [50]

    Revisiting multi-modal emotion learning with broad state space models and probability-guidance fusion

    Shou, Y., Meng, T., Zhang, F., Yin, N., Li, K., 2024f. Revisiting multi-modal emotion learning with broad state space models and probability-guidance fusion. arXiv preprint arXiv:2404.17858

    arXiv

  44. [51]

    Graph domain adaptation with dual-branch encoder and two-level alignment for whole slide image-based survival prediction

    Shou, Y., Yan, P., Yuan, X., Cao, X., Zhao, Q., Meng, D., 2024g. Graph domain adaptation with dual-branch encoder and two-level alignment for whole slide image-based survival prediction. arXiv preprint arXiv:2411.14001

    arXiv

  45. [52]

    Dropout:asimplewaytopreventneuralnetworksfrom overfitting

    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdi- nov,R.,2014. Dropout:asimplewaytopreventneuralnetworksfrom overfitting. Journal of Machine Learning Research 15, 1929–1958

    2014

  46. [53]

    Efficient multimodal transformer with dual-level feature restoration for robust multimodal sentiment analysis

    Sun, L., Lian, Z., Liu, B., Tao, J., 2023. Efficient multimodal transformer with dual-level feature restoration for robust multimodal sentiment analysis. IEEE Transactions on Affective Computing

    2023

  47. [54]

    Missing modalities imputation via cascaded residual autoencoder, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Tran, L., Liu, X., Zhou, J., Jin, R., 2017. Missing modalities imputation via cascaded residual autoencoder, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1405–1414

    2017

  48. [55]

    Multimodal transformer for unaligned mul- timodal language sequences, in: Proceedings of the 57th Conference of the Association for Computational Linguistics, pp

    Tsai, Y.H.H., Bai, S., Liang, P.P., Kolter, J.Z., Morency, L.P., Salakhutdinov, R., 2019. Multimodal transformer for unaligned mul- timodal language sequences, in: Proceedings of the 57th Conference of the Association for Computational Linguistics, pp. 6558–6569

    2019

  49. [56]

    Attention is all you need, in: Proceedings of the Advances in Neural Information Processing Systems, pp

    Vaswani,A.,Shazeer,N.,Parmar,N.,Uszkoreit,J.,Jones,L.,Gomez, A.N., Kaiser, L., Polosukhin, I., 2017. Attention is all you need, in: Proceedings of the Advances in Neural Information Processing Systems, pp. 5998–6008

    2017

  50. [57]

    Ex- tracting and composing robust features with denoising autoencoders, in: Proceedings of the 25th international conference on Machine learning, pp

    Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.A., 2008. Ex- tracting and composing robust features with denoising autoencoders, in: Proceedings of the 25th international conference on Machine learning, pp. 1096–1103

    2008

  51. [58]

    Partialmulti-view clustering via consistent gan, in: IEEE International Conference on Data Mining (ICDM), pp

    Wang,Q.,Ding,Z.,Tao,Z.,Gao,Q.,Fu,Y.,2018. Partialmulti-view clustering via consistent gan, in: IEEE International Conference on Data Mining (ICDM), pp. 1290–1295

    2018

  52. [59]

    On deep multi- viewrepresentationlearning,in:Internationalconferenceonmachine learning, PMLR

    Wang, W., Arora, R., Livescu, K., Bilmes, J., 2015. On deep multi- viewrepresentationlearning,in:Internationalconferenceonmachine learning, PMLR. pp. 1083–1092

    2015

  53. [61]

    Emotion recognitioninconversationbasedonadynamiccomplementarygraph convolutional network

    Yang, Z., Li, X., Cheng, Y., Zhang, T., Wang, X., 2024. Emotion recognitioninconversationbasedonadynamiccomplementarygraph convolutional network. IEEE Transactions on Affective Computing

    2024

  54. [62]

    Prediction model of dow jones index based on lstm-adaboost, in: 2021 International Conference on Communications, Information System and Computer Engineering (CISCE), IEEE

    Ying, R., Shou, Y., Liu, C., 2021. Prediction model of dow jones index based on lstm-adaboost, in: 2021 International Conference on Communications, Information System and Computer Engineering (CISCE), IEEE. pp. 808–812

    2021

  55. [63]

    Transformer-based feature reconstruction network for robust multimodal sentiment analysis, in: Proceedings of the 29th ACM International Conference on Multime- dia, pp

    Yuan, Z., Li, W., Xu, H., Yu, W., 2021. Transformer-based feature reconstruction network for robust multimodal sentiment analysis, in: Proceedings of the 29th ACM International Conference on Multime- dia, pp. 4400–4407

    2021

  56. [64]

    Multimodal sentiment intensity analysis in videos: Facial gestures and verbal messages

    Zadeh, A., Zellers, R., Pincus, E., Morency, L.P., 2016. Multimodal sentiment intensity analysis in videos: Facial gestures and verbal messages. IEEE Intelligent Systems 31, 82–88

    2016

  57. [65]

    Zadeh, A.B., Liang, P.P., Poria, S., Cambria, E., Morency, L.P.,

  58. [66]

    Deep partial multi-view learning

    Zhang, C., Cui, Y., Han, Z., Zhou, J.T., Fu, H., Hu, Q., 2022. Deep partial multi-view learning. IEEE Transactions on Pattern Analysis & Machine Intelligence 44, 2402–2415

    2022

  59. [67]

    A multi-level alignment and cross-modal unified semantic graph refinement network for con- versational emotion recognition

    Zhang, X., Cui, W., Hu, B., Li, Y., 2024. A multi-level alignment and cross-modal unified semantic graph refinement network for con- versational emotion recognition. IEEE Transactions on Affective Computing . Fu et al.:Preprint submitted to Elsevier Page 16 of 16 Fangze Fu et ...

    2024

  60. [68]

    Zhao, J., Li, R., Jin, Q., 2021. Missing modality imagination net- work for emotion recognition with uncertain missing modalities, in: Proceedingsofthe59thAnnualMeetingoftheAssociationforCom- putational Linguistics and the 11th International Joint Conference on Natural Languag...

    2021

  61. [2018]

    2236–2246

    Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph, in: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2236–2246

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.