AIVA: An AI-based Virtual Companion for Emotion-aware Interaction

Chenxi Li

REVIEW 4 major objections 5 minor 42 references

AIVA: An AI-based Virtual Companion for Emotion-aware Interaction

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AIVA argues that giving an LLM a multimodal sentiment-perception front end lets a virtual companion answer with emotionally aligned language, speech, and animated expressions.

desk verdict The architecture is a reasonable incremental combination, but the impossible ablation row (18.70% accuracy / 67.41% F1) makes the empirical claims unreliable until artifacts and corrections appear. read the letter →

arxiv 2509.03212 v1 pith:PXNIUH6D submitted 2025-09-03 cs.CV

Chenxi Li This is my paper · ORCID

classification cs.CV

keywords multimodalsentimentanalysisvirtualcompanionlargelanguagemodelscross-modalfusiontransformersupervisedcontrastivelearningemotion-awarepromptengineeringtext-to-speechanimatedavatar

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that large language models, left to text alone, miss the emotional content carried by faces and voices, and that a companion agent can recover that content by adding a dedicated multimodal sentiment-perception front end. It proposes AIVA, built around the Multimodal Sentiment Perception Network (MSPN), which fuses image and text with cross-attention, a fusion transformer, and prototype-level supervised contrastive learning. The detected sentiment is inserted into the LLM prompt, and the reply is spoken by a TTS system and acted out by an animated avatar. If the reported results hold, the framework offers a working template for emotion-aware agents in companion robotics, social care, and mental-health support.

What carries the argument

The load-bearing mechanism is the Multimodal Sentiment Perception Network (MSPN), specifically its cross-modal fusion transformer with learnable sentiment prototypes. The prototypes act as category queries that repeatedly attend to the fused image-text token sequence; classification reads from the final prototype layer, and a bidirectional supervised contrastive loss forces prototypes and multimodal representations to share a cluster structure. This is what converts raw pixels and words into the sentiment label that the Emotion-aware Prompt Engineering step hands to the LLM.

What would settle it

Re-running MSPN and the reported baselines on MVSA-Single and MVSA-Multi with the same data splits would settle the comparison; one concrete check is the ablation row 'w/o CAF' on MVSA-Multi, which lists 18.70% accuracy with 67.41% F1, a combination that is implausible for a three-way classification and suggests a data or protocol error that either the paper or a replication would have to explain.

Watch

Extended reading notes

Core claim

The central claim is that multimodal sentiment classification and LLM-driven dialogue can be joined in one agent without retraining the LLM. MSPN encodes images and text with a Vision Transformer and BERT, lets each modality attend to the other through cross-attention, then passes the combined tokens through a cross-modal fusion transformer whose learnable sentiment prototypes extract category-level emotion signals. A supervised contrastive loss aligns prototypes and multimodal representations in both directions, and the model is pretrained on TumEmo and fine-tuned on the MVSA datasets. On those benchmarks the paper reports MSPN surpassing all compared baselines, with 74.25% accuracy and 72.84% F1 on MVSA-Single and 73.48% accuracy and 70.01% F1 on MVSA-Multi, plus 81.81% accuracy on TumEmo. The paper's contribution is therefore a complete pipeline from raw multimodal input to empathetic response, with the sentiment module doing the perception work that the LLM alone cannot.

Load-bearing premise

The reported experimental numbers are accurate and were produced under a consistent evaluation protocol, since the paper's empirical claims about MSPN rest entirely on the tables.

Editorial extensions

If this is right

Emotion-aware agents can be built on top of existing LLMs by prepending a sentiment cue to the prompt rather than retraining the dialogue model.
Multimodal sentiment classification improves when sentiment prototypes are contrastively aligned with fused representations, as the reported gains over MGNNS and other baselines indicate.
Removing any component of MSPN, whether cross-attention fusion, the fusion transformer, or supervised contrastive learning, lowers accuracy and F1, suggesting each part contributes to the result.
Expressive output matters to the claimed effect: TTS and the animated avatar give the generated empathy a visible and audible channel, not just a textual one.
Pretraining on a large image-text emotion corpus and then fine-tuning on smaller labeled sets is the protocol that yields the best reported numbers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural next test would be to ablate the sentiment prefix itself, comparing the full AIVA against an LLM prompted without any emotional label, and to measure human-rated empathy rather than classification accuracy.
The prototype-contrastive idea is not restricted to still images and text; the same machinery could ingest audio prosody or gesture features and feed continuous valence-arousal values instead of discrete labels.
Because the sentiment cue is a discrete category, the agent's empathy is bounded by the emotion taxonomy; subtle or mixed emotional states may require finer-grained or continuous representations.
The system's perception module and generation module are loosely coupled, so failures in emotion recognition would degrade the prompt cue but not the LLM itself, making the framework easy to upgrade as better sentiment models appear.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Here is the quick take: the paper has a coherent, mildly novel architecture (MSPN) wrapped inside a broader AIVA companion framework, and the writing is clear. But the empirical core is not trustworthy as presented. The Table 3 ablation row 'w/o CAF' on MVSA-Multi reports 18.70% accuracy with 67.41% F1; that pair cannot happen for a three-class problem. The reader's stress-test is correct: with accuracy that low, F1 is far below 0.67 under any reasonable averaging. One of the two numbers is wrong, and since the same protocol produced all tables, the headline 'MSPN outperforms all baselines' is now in question.

What's actually new: combining cross-attention fusion, a fusion transformer with learnable sentiment prototypes, and bidirectional supervised contrastive loss into one sentiment classifier. That specific combination is not in the cited baselines. The EPE prompt template and GPT-SoVITS/Live2D integration is a reasonable practical framework, though it's not evaluated quantitatively; we only get two qualitative examples. The writing is clear and the related work is adequate.

Soft spots, in order of severity: - The impossible ablation row. This is load-bearing. The authors need to correct it or explain it, and release code/data/splits/seeds so the rest of the numbers can be checked. - No error bars or repeated runs anywhere; the single-run table is thin evidence for a 0.5-1 point win over MGNNS. - The AIVA interaction claims are only illustrated, not measured. No user study, no automated empathy metric. - The TumEmo table has no baselines, only MSPN, so it's a sanity check rather than a comparison.

My verdict: the architecture deserves a look, but the paper in its current form is not ready. The central empirical claim depends entirely on tables that include at least one impossible number. This is exactly the kind of paper that should go to peer review with a request for artifacts and a corrected ablation table; a serious referee could get it into shape. I would not cite it until the numbers are fixed.

Recommendation: send it to review, but make the authors reproduce the ablation and release code before acceptance.

Referee Report

4 major / 5 minor

Summary. The paper proposes AIVA, an LLM-driven virtual companion that extracts sentiment cues from paired text and images with a Multimodal Sentiment Perception Network (MSPN), feeds the predicted emotion into an LLM through an emotion-aware prompt template (EPE), and renders responses via TTS and a Live2D avatar. MSPN combines a Vision Transformer and BERT with cross-attention fusion, a transformer with learnable sentiment prototypes, and bidirectional supervised contrastive learning. The empirical section reports MSPN accuracy/F1 on MVSA-Single, MVSA-Multi, and TumEmo, compares MSPN with six baselines on MVSA, and presents ablations of CAF, CMFT, SCL, and the contrastive-loss weight lambda, plus qualitative examples of the full system.

Significance. If the reported numbers are trustworthy, the contribution is a competent integration of known components (cross-attention, sentiment prototypes, supervised contrastive learning, and prompt-based emotion injection) into a practical LLM-based companion. The MSPN design does not introduce a fundamentally new mechanism, but the systems-level combination and the benchmark comparisons could be useful to the multimodal affective computing and HCI communities. The paper does not provide code, data splits, seeds, or a detailed evaluation protocol, so the empirical claims are not independently reproducible from the manuscript; this is the main limitation of the contribution as presented.

major comments (4)

[Ablation Study, Table 3] The row 'w/o CAF' on MVSA-Multi reports Accuracy 18.70% and F1 67.41%. For a three-class sentiment problem these two numbers are internally inconsistent under the standard macro-F1 definition: with balanced classes, a total true-positive count of 18.7% of the test set caps macro-F1 at roughly 0.3, and class imbalance does not rescue the pair because a high-F1 majority class would necessarily produce many correct predictions, raising accuracy well above 18.7%. This anomaly makes the ablation claim that removing CAF collapses MVSA-Multi accuracy unsupported and casts doubt on whether all rows in Tables 1, 3, and 4 were produced under the same protocol. Please correct the row or provide the confusion matrix, class distribution, and metric definition that make the reported pair possible.
[Experiment Setup; Tables 1, 3, 4] The paper reports no train/validation/test split, no number of random seeds or runs, no standard deviations or significance tests, and no code. The baseline numbers in Table 1 may come from prior papers under different protocols, and the text does not state whether the baselines were retrained with the same preprocessing, pretraining, and split as MSPN. The central claim that 'MSPN outperforms all baselines' cannot be evaluated without these details. The authors should release code, data splits, seeds, and metric definitions, or at minimum provide the exact protocol and per-run results.
[Table 2 and Experiment Setup] The TumEmo result (81.81% accuracy) is presented without any baselines or comparison, and the relationship between TumEmo 'pretraining' and the MVSA results is unclear: it is not stated whether the MSPN rows in Tables 1, 3, and 4 were initialized with TumEmo pretraining, nor whether the comparison methods had access to the same pretraining. If only MSPN is pretrained on TumEmo, the reported gains may be due to additional data rather than the proposed architecture. The transfer protocol must be described explicitly.
[Qualitative Results of AIVA; Conclusion] The empathetic interaction capability of AIVA is supported only by two screenshots in Figure 8; there is no human evaluation, automatic empathy/emotion-consistency metric, or comparison with a text-only LLM baseline. The abstract and conclusion claim that AIVA enables 'emotionally aligned and animated HCI,' but this claim is not substantiated by any measurement. At minimum, the authors should report a small user study or an automatic metric on response emotion alignment, and should temper the claims to match the evidence provided.

minor comments (5)

[Experiment Setup] The sentence 'we evaluate the proposed MSPN on on sev...' contains a duplicated preposition and an incomplete word; the paragraph should be rewritten.
[Eq. (1)] Equation (1) has an extra closing parenthesis in the first attention formula and does not define the projection matrices or d_k; please clarify the notation and the resulting token dimensions in Z_i^0.
[Eqs. (7)-(9)] The positive set P(i) and the anchor set A(i) in the supervised contrastive losses are not defined; the text should state that positives are samples/prototypes of the same sentiment class and whether the anchor itself is excluded.
[Figure 3] Figure 3 is not referenced in the main text; either cite it where the MSPN architecture is introduced or remove it.
[Conclusion] The conclusion refers to 'human-robot interactions' while the rest of the paper discusses human-computer interaction; the terminology should be made consistent.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the reported results are empirical comparisons against external benchmark labels, with no derivation step that reduces to its own input.

full rationale

The paper's central claims are empirical: MSPN is compared against fixed benchmark datasets (MVSA-Single, MVSA-Multi, TumEmo) with externally provided labels, and the AIVA framework is demonstrated through qualitative examples. No formal derivation chain is present in which an output quantity is defined in terms of the quantity it is claimed to predict. The hyperparameter lambda in Eq. 9 is tuned by sweeping values and reporting the best configuration (Table 4); this is standard hyperparameter selection, not a fitted parameter being renamed as a prediction, because the reported accuracies are still measured against the same external labels rather than being constructed from the sweep. The paper does not rely on any self-citation, uniqueness theorem, or prior-work ansatz to ground its architecture choices. The only notable anomaly is Table 3's 'w/o CAF' row on MVSA-Multi, which reports Accuracy 18.70% with F1-Score 67.41%; that pair is internally implausible for a three-class sentiment problem. However, this is a data-integrity or reporting concern, not circularity, because it does not exhibit the specific reduction required by the circularity definition: no equation or fitted value is shown to equal its own input by construction. Under the hard rules, correctness and reproducibility concerns belong outside the circularity score, so the appropriate finding is no significant circularity with score 0.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The method adds one tuned hyperparameter (lambda) and several design choices. The main external inputs are the pretrained encoders and dataset labels. No new physical or formal entities are postulated.

free parameters (1)

lambda (contrastive loss weight) = 1.0
The paper sweeps lambda in {0.5, 1.0, 1.5, 2.0} and selects 1.0 as the best on the MVSA validation sets (Table 4). This is a tuned hyperparameter, not a derived constant.

assumptions (4)

domain assumption MVSA and TumEmo ground-truth labels are correct and consistent.
All training and evaluation use these labels as supervision and as the contrastive-learning positives (Equations 6-8).
domain assumption The evaluation uses the same train/test splits as the cited baselines.
The paper does not describe the split protocol; if splits differ, the accuracy comparisons in Table 1 are not fair.
domain assumption Pretrained ViT and BERT features carry sufficient sentiment information.
The method relies on these pretrained encoders for all visual and textual representations; no analysis of encoder sufficiency is given.
domain assumption Supervised contrastive loss and sentiment prototypes improve sentiment classification.
This is the design hypothesis behind Equations 7-9; the only evidence is the ablation table, which contains a suspicious entry.

how reviews work

0 comments

Cite this review

Pith. "Pith review of AIVA: An AI-based Virtual Companion for Emotion-aware Interaction." pith.science (2026). https://pith.science/paper/PXNIUH6D

@misc{pith2026250903212,
  author       = {Pith},
  title        = {Pith review of: AIVA: An AI-based Virtual Companion for Emotion-aware Interaction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PXNIUH6D}},
  note         = {Machine review of arXiv:2509.03212}
}

read the original abstract

Recent advances in Large Language Models (LLMs) have significantly improved natural language understanding and generation, enhancing Human-Computer Interaction (HCI). However, LLMs are limited to unimodal text processing and lack the ability to interpret emotional cues from non-verbal signals, hindering more immersive and empathetic interactions. This work explores integrating multimodal sentiment perception into LLMs to create emotion-aware agents. We propose \ours, an AI-based virtual companion that captures multimodal sentiment cues, enabling emotionally aligned and animated HCI. \ours introduces a Multimodal Sentiment Perception Network (MSPN) using a cross-modal fusion transformer and supervised contrastive learning to provide emotional cues. Additionally, we develop an emotion-aware prompt engineering strategy for generating empathetic responses and integrate a Text-to-Speech (TTS) system and animated avatar module for expressive interactions. \ours provides a framework for emotion-aware agents with applications in companion robotics, social care, mental health, and human-centered AI.

Figures

Figures reproduced from arXiv: 2509.03212 by the authors.

**Figure 2.** The proposed AIVA framework for empathetic HCI, integrating multimodal sentiment perception, LLM-driven empathetic responses, and expressive feedback through TTS and animated avatars. visual tokens. In parallel, the textual encoder (e.g., BERT) encodes the textual representations as Ti = {t1, t2, . . . , tL}, where t ∈ R 768 and L denotes the length of textual tokens. These tokens are then processed by Cross Attent… view at source ↗

**Figure 4.** Here, we introduce a group of learnable sentiment [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗

Figures from the paper (6 more)

**Figure 3.** Figure 3: The MSPN architecture, where visual and textual representations are fused using Cross-attention and cross-modal [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]

**Figure 4.** Figure 4: The designed cross-modal fusion transformer. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: The designed prompt template. This approach adapts the sentiment classification results as a prefix in the prompt, guiding the LLM to generate emotionally aligned responses. By embedding sentiment information, the model is able to recognize the user’s emotions and r…

**Figure 6.** Figure 6: The t-SNE results of the learned sentiment proto [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

**Figure 7.** Figure 7: The qualitative examples of the MSPN. tively combines multimodal sentiment perception, significantly improving sentiment classification performance. Pretraining Results on TumEmo [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]

**Figure 8.** Figure 8: The qualitative examples of the AIVA. Conclusion In this work, we proposed AIVA, a framework that integrates multimodal sentiment perception with LLMs to enable emotionally intelligent human-robot interactions. By combining the Multimodal Sentiment Perception Network…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 17 canonical work pages

[1]

Allemang, D.; and Sequeda, J. 2024. Increasing the llm accuracy for question answering: Ontologies to the rescue! arXiv preprint arXiv:2405.11706

arXiv 2024
[2]

M.; Meng, R.; Liu, Y.; Zhou, Y.; and Yavuz, S

Bhat, M. M.; Meng, R.; Liu, Y.; Zhou, Y.; and Yavuz, S. 2023. Investigating answerability of llms for long-form question answering. arXiv preprint arXiv:2309.08210

arXiv 2023
[3]

Das, R.; and Singh, T. D. 2023. Multimodal sentiment analysis: a survey of methods, trends, and challenges. ACM Computing Surveys, 55(13s): 1--38

work page 2023
[4]

Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), 4171--4186

2019
[5]

Du, Y.; Liu, Y.; Peng, Z.; and Jin, X. 2022. Gated attention fusion network for multimodal sentiment classification. Knowledge-Based Systems, 240: 108107

work page 2022
[6]

Du, Z.; Qian, Y.; Liu, X.; Ding, M.; Qiu, J.; Yang, Z.; and Tang, J. 2021. Glm: General language model pretraining with autoregressive blank infilling. arXiv preprint arXiv:2103.10360

arXiv 2021
[7]

Haltaufderheide, J.; and Ranisch, R. 2024. The ethics of ChatGPT in medicine and healthcare: a systematic review on Large Language Models (LLMs). NPJ digital medicine, 7(1): 183

work page 2024
[8]

Kim, W.; Son, B.; and Kim, I. 2021. Vilt: Vision-and-language transformer without convolution or region supervision. In International conference on machine learning, 5583--5594. PMLR

2021

Show all 42 references

[9]

Kumar, P. 2024. Large language models (LLMs): survey, technical frameworks, and future challenges. Artificial Intelligence Review, 57(10): 260

2024
[10]

Li, J.; Li, D.; Savarese, S.; and Hoi, S. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, 19730--19742. PMLR

2023
[11]

Li, J.; Li, D.; Xiong, C.; and Hoi, S. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning, 12888--12900. PMLR

2022
[12]

Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2023. Visual instruction tuning. Advances in neural information processing systems, 36: 34892--34916

2023
[13]

McKinzie, B.; Gan, Z.; Fauconnier, J.-P.; Dodge, S.; Zhang, B.; Dufter, P.; Shah, D.; Du, X.; Peng, F.; Belyi, A.; et al. 2024. Mm1: methods, analysis and insights from multimodal llm pre-training. In European Conference on Computer Vision, 304--323. Springer

2024
[14]

Niu, T.; Zhu, S.; Pang, L.; and El Saddik, A. 2016. Sentiment analysis on multi-view social data. In International conference on multimedia modeling, 15--27. Springer

2016
[15]

Patil, R.; and Gudivada, V. 2024. A review of current trends, techniques, and challenges in large language models (llms). Applied Sciences, 14(5): 2074

2024
[16]

Peng, J.; Wu, T.; Zhang, W.; Cheng, F.; Tan, S.; Yi, F.; and Huang, Y. 2023. A fine-grained modal label-based multi-stage network for multimodal sentiment analysis. Expert Systems with Applications, 221: 119721

2023
[17]

Poria, S.; Cambria, E.; Bajpai, R.; and Hussain, A. 2017. A review of affective computing: From unimodal analysis to multimodal fusion. Information fusion, 37: 98--125

2017
[18]

W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PmLR

2021
[19]

Radford, A.; Narasimhan, K.; Salimans, T.; Sutskever, I.; et al. 2018. Improving language understanding by generative pre-training

2018
[20]

Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; and Liu, P. J. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140): 1--67

2020
[21]

Ramachandram, D.; and Taylor, G. W. 2017. Deep multimodal learning: A survey on recent advances and trends. IEEE signal processing magazine, 34(6): 96--108

2017
[22]

RVC-Boss. 2025. GPT-SoVITS: 1 min voice data can also be used to train a good TTS model! (few shot voice cloning)

2025
[23]

Soleymani, M.; Garcia, D.; Jou, B.; Schuller, B.; Chang, S.-F.; and Pantic, M. 2017. A survey of multimodal sentiment analysis. Image and Vision Computing, 65: 3--14

2017
[24]

Taori, R.; Gulrajani, I.; Zhang, T.; Dubois, Y.; Li, X.; Guestrin, C.; Liang, P.; and Hashimoto, T. B. 2023. Alpaca: A strong, replicable instruction-following model. Stanford Center for Research on Foundation Models. https://crfm. stanford. edu/2023/03/13/alpaca. html, 3(6): 7

2023
[25]

J.; Ting, D

Thirunavukarasu, A. J.; Ting, D. S. J.; Elangovan, K.; Gutierrez, L.; Tan, T. F.; and Ting, D. S. W. 2023. Large language models in medicine. Nature medicine, 29(8): 1930--1940

2023
[26]

Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozi \`e re, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

2023 arXiv
[27]

Wang, K.; Duan, F.; Wang, S.; Li, P.; Xian, Y.; Yin, C.; Rong, W.; and Xiong, Z. 2023. Knowledge-driven cot: Exploring faithful reasoning in llms for knowledge-intensive question answering. arXiv preprint arXiv:2308.13259

2023 arXiv
[28]

Xu, N. 2017. Analyzing multimodal public sentiment based on hierarchical semantic attentional network. In 2017 IEEE international conference on intelligence and security informatics (ISI), 152--154. IEEE

2017
[29]

Xu, N.; and Mao, W. 2017. Multisentinet: A deep semantic network for multimodal sentiment analysis. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, 2399--2402

2017
[30]

Xu, N.; Mao, W.; and Chen, G. 2018. A co-memory network for multimodal sentiment analysis. In The 41st international ACM SIGIR conference on research & development in information retrieval, 929--932

2018
[31]

Xu, P.; Zhu, X.; and Clifton, D. A. 2023. Multimodal learning with transformers: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(10): 12113--12132

2023
[32]

K.; and Wang, D

Xu, X.; Yao, B.; Dong, Y.; Gabriel, S.; Yu, H.; Hendler, J.; Ghassemi, M.; Dey, A. K.; and Wang, D. 2024. Mental-llm: Leveraging large language models for mental health prediction via online text data. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Tech...

2024
[33]

Yang, J.; Jin, H.; Tang, R.; Han, X.; Feng, Q.; Jiang, H.; Zhong, S.; Yin, B.; and Hu, X. 2024. Harnessing the power of llms in practice: A survey on chatgpt and beyond. ACM Transactions on Knowledge Discovery from Data, 18(6): 1--32

2024
[34]

Yang, X.; Feng, S.; Wang, D.; and Zhang, Y. 2020. Image-text multimodal emotion classification via multi-view attentional network. IEEE Transactions on Multimedia, 23: 4014--4026

2020
[35]

Yang, X.; Feng, S.; Zhang, Y.; and Wang, D. 2021. Multimodal sentiment detection based on multi-channel graph neural networks. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lang...

2021
[36]

Ye, J.; Zhou, J.; Tian, J.; Wang, R.; Zhou, J.; Gui, T.; Zhang, Q.; and Huang, X. 2022. Sentiment-aware multimodal pre-training for multimodal sentiment analysis. Knowledge-Based Systems, 258: 110021

2022
[37]

Yi, Z.; Ouyang, J.; Liu, Y.; Liao, T.; Xu, Z.; and Shen, Y. 2024. A survey on recent advances in llm-based multi-turn dialogue systems. arXiv preprint arXiv:2402.18013

2024 arXiv
[38]

Zhang, S.; Yin, C.; and Yin, Z. 2022. Multimodal sentiment recognition with multi-task learning. IEEE Transactions on Emerging Topics in Computational Intelligence, 7(1): 200--209

2022
[39]

X.; Zhou, K.; Li, J.; Tang, T.; Wang, X.; Hou, Y.; Min, Y.; Zhang, B.; Zhang, J.; Dong, Z.; et al

Zhao, W. X.; Zhou, K.; Li, J.; Tang, T.; Wang, X.; Hou, Y.; Min, Y.; Zhang, B.; Zhang, J.; Dong, Z.; et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223, 1(2)

2023 arXiv
[40]

Zhu, D.; Chen, J.; Shen, X.; Li, X.; and Elhoseiny, M. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592

2023 arXiv
[41]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
[42]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Allemang, D.; and Sequeda, J. 2024. Increasing the llm accuracy for question answering: Ontologies to the rescue! arXiv preprint arXiv:2405.11706

arXiv 2024

[2] [2]

M.; Meng, R.; Liu, Y.; Zhou, Y.; and Yavuz, S

Bhat, M. M.; Meng, R.; Liu, Y.; Zhou, Y.; and Yavuz, S. 2023. Investigating answerability of llms for long-form question answering. arXiv preprint arXiv:2309.08210

arXiv 2023

[3] [3]

Das, R.; and Singh, T. D. 2023. Multimodal sentiment analysis: a survey of methods, trends, and challenges. ACM Computing Surveys, 55(13s): 1--38

work page 2023

[4] [4]

Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), 4171--4186

2019

[5] [5]

Du, Y.; Liu, Y.; Peng, Z.; and Jin, X. 2022. Gated attention fusion network for multimodal sentiment classification. Knowledge-Based Systems, 240: 108107

work page 2022

[6] [6]

Du, Z.; Qian, Y.; Liu, X.; Ding, M.; Qiu, J.; Yang, Z.; and Tang, J. 2021. Glm: General language model pretraining with autoregressive blank infilling. arXiv preprint arXiv:2103.10360

arXiv 2021

[7] [7]

Haltaufderheide, J.; and Ranisch, R. 2024. The ethics of ChatGPT in medicine and healthcare: a systematic review on Large Language Models (LLMs). NPJ digital medicine, 7(1): 183

work page 2024

[8] [8]

Kim, W.; Son, B.; and Kim, I. 2021. Vilt: Vision-and-language transformer without convolution or region supervision. In International conference on machine learning, 5583--5594. PMLR

2021