GSDNet: Revisiting Incomplete Multimodal-Diffusion from Graph Spectrum Perspective for Conversation Emotion Recognition

Cen Chen; Jun Yao; Keqin Li; Tao Meng; Wei Ai; Yuntao Shou

REVIEW 5 major objections 5 minor 32 references

GSDNet: Revisiting Incomplete Multimodal-Diffusion from Graph Spectrum Perspective for Conversation Emotion Recognition

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Eigenvalue-only diffusion recovers missing emotion modalities in conversations.

desk verdict A plausible spectral-diffusion idea for missing-modality recovery, undermined by reproducibility problems and a load-bearing gap about where eigenvectors come from; worth a serious referee but not acceptance as is. read the letter →

arxiv 2506.12325 v1 pith:SBK5MJ4E submitted 2025-06-14 cs.SD cs.CLeess.AS

Yuntao Shou , Jun Yao , Tao Meng , Wei Ai , Cen Chen , Keqin Li This is my paper · ORCID

classification cs.SDcs.CLeess.AS

keywords multimodalemotionrecognitionincompletelearninggraphdiffusionspectraltheoryscore-basedgenerativemodelsmodalityrecoveryeigenvalueperturbationconversation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the right place to run diffusion for graph-structured multimodal data is the spectral space of the adjacency matrix, not the adjacency matrix itself. To recover a missing modality (audio, video, or text) in conversation emotion recognition, GSDNet decomposes that modality's adjacency matrix into eigenvectors and eigenvalues, adds Gaussian noise only to the eigenvalues, and learns to denoise them back. Because the eigenvectors are left untouched, the diffusion process does not destroy the graph's connectivity or local structure, so the reconstructed modality keeps its global topology and spectral features. The authors show on the CMU-MOSI and CMU-MOSEI datasets that this spectral-restricted diffusion outperforms previous graph-diffusion and graph-completion baselines across every modality-missing pattern and missing-rate setting.

What carries the argument

The central object is the eigendecomposition of the adjacency matrix, $A_m = U_m \Lambda_m U_m^T$, where $\Lambda_m$ is the diagonal matrix of eigenvalues. GSDNet runs its forward and reverse stochastic differential equations on $\Lambda_m$ only, adding Gaussian noise to the eigenvalues and leaving $U_m$ fixed, which the paper asserts preserves global topological information and spectral features. Two score networks parameterize the denoising: $s_\theta$ for the missing node features $X_m$ and $s_\phi$ for the missing eigenvalues $\Lambda_m$, both conditioned on observed modalities, and reconstruction modules $D_X$, $D_\Lambda$ refine the sampled output before a GCN fuses all modalities for emotion classification.

What would settle it

Measure the spectral and topological fidelity of the reconstructed adjacency matrices: if, for a missing modality, the eigenvectors $U_m$ used at inference are replaced by eigenvectors inferred from observed modalities alone and emotion recognition accuracy drops to baseline levels, or if the reconstructed graphs' Laplacian spectra diverge from the originals under eigenvalue-only perturbation, the paper's central preservation claim would be falsified.

Watch

Extended reading notes

Core claim

The central claim is that incomplete multimodal learning for conversation emotion recognition can be recast as a graph diffusion problem in spectral space: given the adjacency matrix of a missing modality, $A_m = U_m \Lambda_m U_m^T$, the forward process corrupts only the eigenvalue matrix $\Lambda_m$ with Gaussian noise while holding the eigenvectors $U_m$ fixed, and two score networks, $s_\theta$ for node features and $s_\phi$ for eigenvalues, reverse the noise conditioned on the observed modalities. This yields reconstructed modalities that preserve the original graph's global topology and semantic distribution, and the recovered features, when fused by a GCN, deliver emotion classification accuracy superior to prior incomplete-modality methods, with larger gains as the missing ratio grows.

Load-bearing premise

The load-bearing premise is that the eigenvectors of the missing modality's adjacency matrix are known at inference even though the modality itself is missing, since only eigenvalues are diffused and the matrix $A_m = U_m \Lambda_m U_m^T$ must be reconstructed from those eigenvectors plus the denoised eigenvalues.

Editorial extensions

If this is right

On both CMU-MOSI and CMU-MOSEI, GSDNet reports the best ACC2/F1/ACC7 under every single-modality and multi-modality missing pattern, including a 77.3/77.1/37.3 average across the MOSI missing-modality settings.
The gains enlarge as the missing ratio increases (for example, at a 0.7 missing rate on MOSI, ACC2 remains 77.6 versus 71.9 for the best baseline), implying that eigenvalue-restricted diffusion degrades more gracefully.
Restored modalities from GSDNet overlap the original feature distribution more closely in t-SNE visualizations and interpolation comparisons, supporting the claim that spectral-space diffusion preserves data distribution consistency.
The recovered modalities supply complementary information to the observed ones, and fusing all three via GCN yields higher emotion classification accuracy than any baseline fusion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If eigenvalue-only perturbation truly preserves topology, the same spectral-diffusion design should transfer to other sparse graph generation or imputation tasks such as molecular graph generation, social network completion, or sensor network recovery, where connectivity matters as much as feature values.
A testable extension would condition the forward process on the observed modalities' eigenvectors and let the score network predict the missing modality's eigenvectors too, removing the assumption that $U_m$ is available at inference and measuring how much accuracy is lost when it is absent.
The paper argues that fixed eigenvectors preserve global topology, but it never quantifies this; computing the Laplacian spectral distance between original and reconstructed graphs across missing rates would give a direct, falsifiable measure of the claimed preservation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The core idea here is a real extension: instead of adding Gaussian noise directly to the adjacency matrix, GSDNet diffuses only the eigenvalues in spectral space, conditioning on observed modalities. That's a legitimate combination of existing work (spectral diffusion from Luo et al., incomplete-modality diffusion from IMDer), and it's aimed at a real problem—sparse graph structure is destroyed by naive graph diffusion. The paper does a good job motivating that problem, and the experimental coverage (multiple missing-modal patterns and missing-rate scenarios) is broad. If the reported gains are real, this is useful for the MERC community.

But the soft spots are substantial. First, reproducibility: no code, no error bars, no hyperparameters, no training details. For a state-of-the-art claim, that's a serious omission. Second, Table 2 contains suspicious repeated values: the DiCMoR column for CMU-MOSEI at missing rates 0.0 and 0.1 is identical (78.9/78.7/38.5), which looks like a copy-paste error. That undermines trust in all the numbers. Third, the ablation in Table 3 is confusing: the row labeled "GSDNet w/spectral" is stronger than the row labeled "GSDNet," but the text says removing spectral diffusion degrades performance. The rows look swapped, and as printed the table doesn't support the narrative.

The deepest issue is the stress-test point: the paper never says where the eigenvectors U_m come from for a missing modality at inference. The reverse SDE in Eq. 16 updates X'_m and Lambda'_m but never uses U_m. If eigenvectors genuinely aren't needed, then the "spectral" framing is mostly a reparametrization, and the topological-preservation argument is disconnected from the algorithm. If U_m is taken from some shared graph, the paper must say so. As written, the method can't be run without making an assumption the paper doesn't state.

The theoretical derivation (Eqs. 8–11) also has asserted approximations rather than derivations, and the training target in Eq. 11 doesn't clearly follow from the conditional score in Eq. 9. These are fixable with clearer exposition, but they matter.

Who is this for? Researchers in missing-modality robustness for affective computing, and people working on graph diffusion. It deserves a serious referee because the idea is plausible and the problems are addressable—but the current manuscript is not close to publishable. I'd send it to review, then require code and a clear resolution of the eigenvector gap before any acceptance. If the numbers don't hold up once reproducible, reject.

Referee Report

5 major / 5 minor

Summary. The paper proposes GSDNet, a method for incomplete multimodal emotion recognition in conversations that combines graph neural networks with score-based diffusion models. The key claim is that instead of perturbing the adjacency matrix directly with Gaussian noise, GSDNet perturbs only the eigenvalues of the adjacency matrix in spectral space, leaving eigenvectors untouched, thereby preserving global topology and spectral features. The method is evaluated on CMU-MOSI and CMU-MOSEI for various missing modality and missing ratio scenarios, with reported state-of-the-art accuracy. The paper also provides an ablation study and t-SNE visualizations.

Significance. If the central mechanism were well-defined and the empirical gains were robust, the idea of restricting graph diffusion to spectral space would be a useful contribution to incomplete multimodal learning. The paper identifies a genuine problem—direct adjacency-matrix diffusion can destroy graph topology—and the proposed spectral perturbation is a plausible remedy. However, as written, the method section does not specify how the eigenvectors of a missing modality are obtained at inference, the conditional score derivation in Section 4 contains unjustified approximations, and the reported experiments lack error bars and training details. Until these issues are resolved, the empirical claims cannot be validated, and the spectral-diffusion mechanism is not operational.

major comments (5)

[Sec. 5.2, Eq. (16); Sec. 4, line defining A_m = U_m Λ_m U_m^T] The manuscript never specifies how the eigenvector matrix U_m of a missing modality is obtained at inference. The method assumes that the missing modality's adjacency matrix can be factorized as A_m = U_m Λ_m U_m^T, yet at test time only the observed modalities are available. The reverse SDE in Eq. (16) and the reconstruction modules D_X and D_Λ operate only on X'_a and Λ'_a; U_a does not appear anywhere in the forward pass or the loss. Without a definition of where U_m comes from—whether it is borrowed from a shared conversation graph, estimated online, or computed from the reconstructed features—the method as written cannot be run, and the claimed topology-preservation mechanism is disconnected from the algorithm.
[Sec. 4, Eq. (8)] The approximation p_t(X_m(t)|X_Io(t), X_Io(0)) ≈ p_t(X_m(t)|X_Io(t)) drops the conditioning on the initial observed features inside the integral. This identity is not generally valid: the distribution of the missing modality at time t can depend on the observed modality at time 0 beyond its time-t value, especially when the two modalities are statistically dependent. The same issue affects the eigenvalue equation. This approximation is load-bearing for the derivation of the conditional score in Eq. (9), so the score network is not guaranteed to be estimating the claimed conditional distribution.
[Sec. 4, Eqs. (9)-(11)] The derivation in Eq. (9) concludes that the conditional score ∇ log p_t(X_m(t)|X_Io(0)) can be approximated by the joint score ∇ log p_t([X_m(t); X_Io(t)]). However, the training loss in Eq. (11) uses the target ∇ log p_{t|0}(X_m(t)|X_Io(0)), which is the conditional transition score, not the joint score. These are different quantities, and the paper does not explain how the network s_θ is parameterized to match one or the other. The loss therefore does not follow from the preceding derivation.
[Table 3] The ablation table is internally inconsistent. The row labeled 'GSDNet' reports average accuracies of 75.7/70.6/35.3 on CMU-MOSI and 78.1/77.4/47.4 on CMU-MOSEI, while the row 'GSDNet w/spectral' reports 83.6/83.2/42.3 and 82.8/82.5/51.9. Since the full GSDNet is presented in Tables 1 and 2 with average accuracies of 77.3/77.1/37.3 on MOSI and 77.7/77.6/50.7 on MOSEI, the table appears to have the labels or columns swapped, and the text's statement that 'Removing the frequency diffusion degrades the performance' cannot be verified from the printed table. This undermines the ablation claim.
[Sec. 7, Tables 1 and 2] All quantitative results are reported without error bars, standard deviations, or statistical significance tests, and no training details (optimizer, learning rate, epochs, random seeds) are provided. In the random-missing-ratio experiments (Table 2), GSDNet outperforms the best baseline by up to 4-5 accuracy points at high missing rates, but without variance estimates it is impossible to determine whether these differences are significant. Given that the paper's central claim is state-of-the-art performance, this lack of experimental detail is a serious omission.

minor comments (5)

[Sec. 7.1] The dataset name 'CMU-CMSEI' should be 'CMU-MOSEI'.
[Fig. 2] The equations inside Fig. 2 are corrupted and unreadable in the PDF; the figure should be regenerated with legible formulas.
[Eq. (10)] The second line of Eq. (10) shows '∇_{A_m} log p_t(Λ_Io(t))' where the variable should be Λ_m, and the term should be zero since the second factor does not depend on Λ_m; as printed, the derivation is incorrect.
[Fig. 1 caption] The caption of Fig. 1 is uninformative; it only repeats 'Forward: Reversed:' without explaining the panels.
[References] References [Wang et al., 2023b] and [Wang et al., 2024] appear to refer to the same paper 'Incomplete multimodality-diffused emotion recognition' with different years; please check and consolidate.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; the spectral-diffusion design is evaluated empirically, though the topology-preservation claim is under-supported.

full rationale

Walking the derivation chain, I find no step in which a claimed prediction is equivalent, by construction, to a fitted input or to the method's own definition. The score-matching objectives in Eqs. 3 and 11 are standard score-based diffusion losses: s_theta and s_phi are optimized against conditional score targets derived from the data distribution, and the reconstruction loss in Eq. 17 is a conventional masked-reconstruction objective whose target is the ground-truth missing modality available during training. At test time the missing X_a and Lambda_a are produced by the reverse-time SDE in Eq. 16 rather than looked up or interpolated from the observed inputs, so the reported ACC2/F1/ACC7 improvements are empirical outcomes, not identities. The paper does cite the authors' own spectral-graph line (Ai et al. 2025; Fu et al. 2025), but those citations are used for motivation and related-work context, not to prove the central claim; the reverse-SDE derivation cites standard external work (Song et al.; Jo et al. 2022). The most serious issues are non-circular: the construction of U_m for a missing modality is never specified, and the claim that eigenvalue-only perturbation preserves global topology is asserted rather than derived or measured beyond t-SNE. These are completeness and support gaps that belong in a correctness review, not evidence that the accuracy numbers reduce to the inputs by construction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The model introduces no new physical or mathematical entities, but it depends on several unstated assumptions: the applicability of SGM theory to spectral components, a questionable conditional-score approximation, the availability of eigenvectors for missing modalities, and an unproven topology-preservation property. All four are load-bearing for the claimed results.

free parameters (4)

Beta (loss weight in L_total)
Eq. 20 combines imputation and classification losses; no value or tuning procedure is given.
Diffusion noise schedule sigma_t
The SDE in Eq. 1 and Eq. 15 relies on a noise schedule; not specified.
Number of diffusion steps
The reverse SDE in Eq. 16 requires a discrete step count and step size; not reported.
Common feature dimension and Conv1D kernel sizes
Eq. 12 maps modalities to a common space; target dimension d and kernel sizes l_m are not reported.

assumptions (4)

domain assumption Score-based generative model theory (forward/reverse SDEs, score matching) applies to graph spectral components.
Section 3.1-3.2 adopts standard SGM results without adaptation to the spectral domain.
ad hoc to paper Conditional score approximation pt(Xm(t) | XIo(0)) ≈ integral of pt(Xm(t) | XIo(t)) pt(XIo(t) | XIo(0)) dXIo(t), with the XIo(0) condition dropped inside the integral.
Eq. 8 in Section 4; this is a mean-field-like approximation that is not justified and changes the training target relative to Eq. 11.
ad hoc to paper Eigenvectors U_m of the missing modality adjacency matrix are available during training and inference.
Section 4 defines A_m = U_m Lambda_m U_m^T; if the modality is missing, U_m cannot be recovered from data unless topology is externally known, and the paper does not explain how.
ad hoc to paper Perturbing eigenvalues while fixing eigenvectors preserves global topology and semantic structure.
Section 1 and Section 4 assert this, but no theorem or systematic measurement is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of GSDNet: Revisiting Incomplete Multimodal-Diffusion from Graph Spectrum Perspective for Conversation Emotion Recognition." pith.science (2026). https://pith.science/paper/SBK5MJ4E

@misc{pith2026250612325,
  author       = {Pith},
  title        = {Pith review of: GSDNet: Revisiting Incomplete Multimodal-Diffusion from Graph Spectrum Perspective for Conversation Emotion Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SBK5MJ4E}},
  note         = {Machine review of arXiv:2506.12325}
}

read the original abstract

Multimodal emotion recognition in conversations (MERC) aims to infer the speaker's emotional state by analyzing utterance information from multiple sources (i.e., video, audio, and text). Compared with unimodality, a more robust utterance representation can be obtained by fusing complementary semantic information from different modalities. However, the modality missing problem severely limits the performance of MERC in practical scenarios. Recent work has achieved impressive performance on modality completion using graph neural networks and diffusion models, respectively. This inspires us to combine these two dimensions through the graph diffusion model to obtain more powerful modal recovery capabilities. Unfortunately, existing graph diffusion models may destroy the connectivity and local structure of the graph by directly adding Gaussian noise to the adjacency matrix, resulting in the generated graph data being unable to retain the semantic and topological information of the original graph. To this end, we propose a novel Graph Spectral Diffusion Network (GSDNet), which maps Gaussian noise to the graph spectral space of missing modalities and recovers the missing data according to its original distribution. Compared with previous graph diffusion methods, GSDNet only affects the eigenvalues of the adjacency matrix instead of destroying the adjacency matrix directly, which can maintain the global topological information and important spectral features during the diffusion process. Extensive experiments have demonstrated that GSDNet achieves state-of-the-art emotion recognition performance in various modality loss scenarios.

Figures

Figures reproduced from arXiv: 2506.12325 by the authors.

**Figure 2.** The framework of GSDNet. Given incomplete input data, GSDNet encodes shallow features through 1D-Conv and combines [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Visualization of restored modalities. Avail. indicates available. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: The comparison of interpolation performance under differ [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 29 canonical work pages

[1]

A two-stage mul- timodal emotion recognition model based on graph con- trastive learning

[Aiet al., 2023 ] Wei Ai, FuChen Zhang, Tao Meng, YunTao Shou, HongEn Shao, and Keqin Li. A two-stage mul- timodal emotion recognition model based on graph con- trastive learning. In2023 IEEE 29th International Confer- ence on Parallel and Distributed Systems (ICPADS), pages 397–404. IEEE,

work page 2023
[5]

Sdr-gnn: Spectral domain re- construction graph neural network for incomplete mul- timodal learning in conversational emotion recognition

[Fuet al., 2025 ] Fangze Fu, Wei Ai, Fan Yang, Yuntao Shou, Tao Meng, and Keqin Li. Sdr-gnn: Spectral domain re- construction graph neural network for incomplete mul- timodal learning in conversational emotion recognition. Knowledge-Based Systems, 309:112825,

work page 2025
[6]

Mmgcn: Multimodal fusion via deep graph convolution network for emotion recognition in conversa- tion

[Huet al., 2021 ] Jingwen Hu, Yuchen Liu, Jinming Zhao, and Qin Jin. Mmgcn: Multimodal fusion via deep graph convolution network for emotion recognition in conversa- tion. InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Pro- cessing (Volume 1: Long Pa...

work page 2021
[11]

Gcnet: Graph completion net- work for incomplete multimodal learning in conversation

[Lianet al., 2023 ] Zheng Lian, Lan Chen, Licai Sun, Bin Liu, and Jianhua Tao. Gcnet: Graph completion net- work for incomplete multimodal learning in conversation. IEEE Transactions on pattern analysis and machine intel- ligence, 45(7):8419–8432,

2023
[12]

Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692,

[Liuet al., 2019 ] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692,

arXiv 2019
[14]

Training strategies to handle miss- ing modalities for audio-visual expression recognition

[Parthasarathy and Sundaram, 2020] Srinivas Parthasarathy and Shiva Sundaram. Training strategies to handle miss- ing modalities for audio-visual expression recognition. In Companion Publication of the 2020 International Confer- ence on Multimodal Interaction, pages 400–404,

work page 2020
[16]

Zero-shot text-to-image generation

[Rameshet al., 2021 ] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. InInternational Conference on Machine Learning, pages 8821–8831. Pmlr,

work page 2021
[17]

wav2vec: Unsuper- vised pre-training for speech recognition

[Schneideret al., 2019 ] Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. wav2vec: Unsuper- vised pre-training for speech recognition. InProceedings of the Interspeech, pages 3465–3469,

work page 2019

Show all 32 references

[18]

Conversational emotion recog- nition studies based on graph convolutional neural net- works and a dependent syntactic analysis.Neurocomput- ing, 501:629–639,

[Shouet al., 2022 ] Yuntao Shou, Tao Meng, Wei Ai, Si- han Yang, and Keqin Li. Conversational emotion recog- nition studies based on graph convolutional neural net- works and a dependent syntactic analysis.Neurocomput- ing, 501:629–639,

2022
[19]

A comprehensive survey on multi- modal conversational emotion recognition with deep learning.arXiv preprint arXiv:2312.05735,

[Shouet al., 2023 ] Yuntao Shou, Tao Meng, Wei Ai, Nan Yin, and Keqin Li. A comprehensive survey on multi- modal conversational emotion recognition with deep learning.arXiv preprint arXiv:2312.05735,

2023
[20]

Efficient long- distance latent relation-aware graph neural network for multi-modal emotion recognition in conversations.arXiv preprint arXiv:2407.00119,

[Shouet al., 2024a ] Yuntao Shou, Wei Ai, Jiayi Du, Tao Meng, Haiyan Liu, and Nan Yin. Efficient long- distance latent relation-aware graph neural network for multi-modal emotion recognition in conversations.arXiv preprint arXiv:2407.00119,

arXiv
[21]

Spegcl: Self-supervised graph spectrum con- trastive learning without positive samples.arXiv preprint arXiv:2410.10365,

[Shouet al., 2024b ] Yuntao Shou, Xiangyong Cao, and Deyu Meng. Spegcl: Self-supervised graph spectrum con- trastive learning without positive samples.arXiv preprint arXiv:2410.10365,

arXiv
[22]

Revisiting multi-modal emotion learning with broad state space models and probability- guidance fusion.arXiv preprint arXiv:2404.17858,

[Shouet al., 2024e ] Yuntao Shou, Tao Meng, Fuchen Zhang, Nan Yin, and Keqin Li. Revisiting multi-modal emotion learning with broad state space models and probability- guidance fusion.arXiv preprint arXiv:2404.17858,

arXiv
[23]

Generative modeling by estimating gradients of the data distribution.Advances in Neural Information Processing Systems, 32,

[Song and Ermon, 2019] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution.Advances in Neural Information Processing Systems, 32,

2019
[24]

Improved techniques for training score-based generative models.Advances in Neural Information Processing Sys- tems, 33:12438–12448,

[Song and Ermon, 2020] Yang Song and Stefano Ermon. Improved techniques for training score-based generative models.Advances in Neural Information Processing Sys- tems, 33:12438–12448,

2020
[25]

Score-based generative modeling through stochas- tic differential equations

[Songet al., ] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochas- tic differential equations. InInternational Conference on Learning Representations. [Tranet al., 2017 ] Luan Tran,...

2017
[26]

Multimodal transformer for un- aligned multimodal language sequences

[Tsaiet al., 2019 ] Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J Zico Kolter, Louis-Philippe Morency, and Ruslan Salakhutdinov. Multimodal transformer for un- aligned multimodal language sequences. InProceedings of the 57th Conference of the Association for Computa- tio...

2019
[27]

Incomplete multimodality-diffused emotion recognition

[Wanget al., 2024 ] Yuanzhi Wang, Yong Li, and Zhen Cui. Incomplete multimodality-diffused emotion recognition. Advances in Neural Information Processing Systems, 36,

2024
[28]

Multimodal sentiment in- tensity analysis in videos: Facial gestures and verbal mes- sages.IEEE Intelligent Systems, 31(6):82–88,

[Zadehet al., 2016 ] Amir Zadeh, Rowan Zellers, Eli Pincus, and Louis-Philippe Morency. Multimodal sentiment in- tensity analysis in videos: Facial gestures and verbal mes- sages.IEEE Intelligent Systems, 31(6):82–88,

2016
[30]

Deep partial multi-view learning.IEEE transactions on pattern analysis and machine intelligence, 44(5):2402– 2415,

[Zhanget al., 2020 ] Changqing Zhang, Yajie Cui, Zongbo Han, Joey Tianyi Zhou, Huazhu Fu, and Qinghua Hu. Deep partial multi-view learning.IEEE transactions on pattern analysis and machine intelligence, 44(5):2402– 2415,

2020
[31]

Learning unseen modality interaction.Ad- vances in Neural Information Processing Systems, 36,

[Zhanget al., 2024 ] Yunhua Zhang, Hazel Doughty, and Cees Snoek. Learning unseen modality interaction.Ad- vances in Neural Information Processing Systems, 36,

2024
[32]

Missing modality imagination network for emotion recog- nition with uncertain missing modalities

[Zhaoet al., 2021 ] Jinming Zhao, Ruichen Li, and Qin Jin. Missing modality imagination network for emotion recog- nition with uncertain missing modalities. InProceedings of the 59th Annual Meeting of the Association for Compu- tational Linguistics and the 11th International J...

2021
[2016]

Multimodal language analysis in the wild: Cmu- mosei dataset and interpretable dynamic fusion graph

[Zadehet al., 2018 ] AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, and Louis-Philippe Morency. Multimodal language analysis in the wild: Cmu- mosei dataset and interpretable dynamic fusion graph. In Proceedings of the 56th Annual Meeting of the Association...

2018
[2017]

Score-based generative modeling of graphs via the system of stochastic differential equations

[Joet al., 2022 ] Jaehyeong Jo, Seul Lee, and Sung Ju Hwang. Score-based generative modeling of graphs via the system of stochastic differential equations. InInter- national conference on machine learning, pages 10362– 10383. PMLR,

2022
[2018]

De- coupled multimodal distilling for emotion recognition

[Liet al., 2023 ] Yong Li, Yuanzhi Wang, and Zhen Cui. De- coupled multimodal distilling for emotion recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6631–6640,

2023
[2019]

Fast graph generation via spectral diffusion.IEEE Transactions on Pattern Analysis and Machine Intelligence,

[Luoet al., 2023 ] Tianze Luo, Zhanfeng Mo, and Sinno Jialin Pan. Fast graph generation via spectral diffusion.IEEE Transactions on Pattern Analysis and Machine Intelligence,

2023
[2020]

Found in translation: Learning robust joint representations by cyclic translations between modalities

[Phamet al., 2019 ] Hai Pham, Paul Pu Liang, Thomas Manzini, Louis-Philippe Morency, and Barnab ´as P ´oczos. Found in translation: Learning robust joint representations by cyclic translations between modalities. InProceed- ings of the AAAI conference on artificial intelligenc...

2019
[2021]

Densely con- nected convolutional networks

[Huanget al., 2017 ] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely con- nected convolutional networks. InProceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition, pages 4700–4708,

2017
[2022]

Glow: Generative flow with invertible 1x1 con- volutions.Advances in Neural Information Processing Systems, 31,

[Kingma and Dhariwal, 2018] Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 con- volutions.Advances in Neural Information Processing Systems, 31,

2018
[2023]

Der-gcn: Dialog and event relation-aware graph con- volutional neural network for multimodal dialog emotion recognition.IEEE Transactions on Neural Networks and Learning Systems,

[Aiet al., 2024 ] Wei Ai, Yuntao Shou, Tao Meng, and Keqin Li. Der-gcn: Dialog and event relation-aware graph con- volutional neural network for multimodal dialog emotion recognition.IEEE Transactions on Neural Networks and Learning Systems,

2024
[2024]

Revisiting multi- modal emotion recognition in conversation from the per- spective of graph spectrum

[Aiet al., 2025 ] Wei Ai, Fuchen Zhang, Yuntao Shou, Tao Meng, Haowen Chen, and Keqin Li. Revisiting multi- modal emotion recognition in conversation from the per- spective of graph spectrum. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 114...

2025
[2025]

Lggnet: Learn- ing from local-global-graph representations for brain– computer interface.IEEE Transactions on Neural Net- works and Learning Systems,

[Dinget al., 2023 ] Yi Ding, Neethu Robinson, Chengxuan Tong, Qiuhao Zeng, and Cuntai Guan. Lggnet: Learn- ing from local-global-graph representations for brain– computer interface.IEEE Transactions on Neural Net- works and Learning Systems,

2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

A two-stage mul- timodal emotion recognition model based on graph con- trastive learning

[Aiet al., 2023 ] Wei Ai, FuChen Zhang, Tao Meng, YunTao Shou, HongEn Shao, and Keqin Li. A two-stage mul- timodal emotion recognition model based on graph con- trastive learning. In2023 IEEE 29th International Confer- ence on Parallel and Distributed Systems (ICPADS), pages 397–404. IEEE,

work page 2023

[2] [5]

Sdr-gnn: Spectral domain re- construction graph neural network for incomplete mul- timodal learning in conversational emotion recognition

[Fuet al., 2025 ] Fangze Fu, Wei Ai, Fan Yang, Yuntao Shou, Tao Meng, and Keqin Li. Sdr-gnn: Spectral domain re- construction graph neural network for incomplete mul- timodal learning in conversational emotion recognition. Knowledge-Based Systems, 309:112825,

work page 2025

[3] [6]

Mmgcn: Multimodal fusion via deep graph convolution network for emotion recognition in conversa- tion

[Huet al., 2021 ] Jingwen Hu, Yuchen Liu, Jinming Zhao, and Qin Jin. Mmgcn: Multimodal fusion via deep graph convolution network for emotion recognition in conversa- tion. InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Pro- cessing (Volume 1: Long Pa...

work page 2021

[4] [11]

Gcnet: Graph completion net- work for incomplete multimodal learning in conversation

[Lianet al., 2023 ] Zheng Lian, Lan Chen, Licai Sun, Bin Liu, and Jianhua Tao. Gcnet: Graph completion net- work for incomplete multimodal learning in conversation. IEEE Transactions on pattern analysis and machine intel- ligence, 45(7):8419–8432,

2023

[5] [12]

Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692,

[Liuet al., 2019 ] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692,

arXiv 2019

[6] [14]

Training strategies to handle miss- ing modalities for audio-visual expression recognition

[Parthasarathy and Sundaram, 2020] Srinivas Parthasarathy and Shiva Sundaram. Training strategies to handle miss- ing modalities for audio-visual expression recognition. In Companion Publication of the 2020 International Confer- ence on Multimodal Interaction, pages 400–404,

work page 2020

[7] [16]

Zero-shot text-to-image generation

[Rameshet al., 2021 ] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. InInternational Conference on Machine Learning, pages 8821–8831. Pmlr,

work page 2021

[8] [17]

wav2vec: Unsuper- vised pre-training for speech recognition

[Schneideret al., 2019 ] Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. wav2vec: Unsuper- vised pre-training for speech recognition. InProceedings of the Interspeech, pages 3465–3469,

work page 2019