Pith. sign in

REVIEW 3 major objections 6 minor 60 references

MGHFT: Multi-Granularity Hierarchical Fusion Transformer for Cross-Modal Sticker Emotion Recognition

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Sticker emotion recognition improves when four MLLM-written views of intent, style, roles, and details are fused stage by stage into a vision transformer.

desk verdict A credible sticker-emotion architecture with MLLM-generated multi-view text, but the headline gain over prior work is confounded by an unmatched text source and single-run results. read the letter →

arxiv 2507.18929 v1 pith:V37PAKAH submitted 2025-07-25 cs.CV cs.AI

Jian Chen , Yuxuan Hu , Haifeng Lu , Wei Wang , Min Yang , Chengming Li , Xiping Hu This is my paper · ORCID
classification cs.CVcs.AI
keywords stickeremotionrecognitionmultimodalfusionmulti-granularityhierarchicallargelanguagemodelspyramidvisiontransformercontrastivelearningaffectivecomputing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that sticker emotion recognition fails with pre-trained vision-language models alone because emotions in stickers depend on implicit, multi-view cues such as intent, style, and character details. It proposes MGHFT, which uses a Multimodal Large Language Model to generate four short textual descriptions of each sticker and injects each description into a different stage of a pyramid vision transformer through attention and contrastive losses. On SER30K the method reaches 73.31% accuracy and 72.52% F1, exceeding the best prior sticker emotion model TGCA-PVT by 2.3 and 2.2 points and the best pre-trained visual model BLIP2 by 4.0 and 5.4 points. On the smaller MET-MEME benchmark it also edges the prior best with 35.13% accuracy. A sympathetic reader would take the paper's point to be that explicit, decomposed language descriptions can guide visual feature learning at multiple scales better than image-only models or generic image-text alignment.

What carries the argument

The load-bearing object is a multi-granularity hierarchical fusion loop built on the Pyramid Vision Transformer (PVT). A frozen BERT encoder turns four MLLM-generated descriptions into text features; at stage $i$, description $T_i$ is injected with local key tokens through a Soft-Fusion attention step $V'_l = V_l + \mathrm{Softmax}(V_l T_i^{\top})T_i$, while the same description aligns the global CLS token through a contrastive loss and the multi-level cross-entropy (MLCE) distribution-matching loss. A Text-Guided Fusion Attention then applies two cross-attention passes, first letting text attend to visual features and then letting the result attend back to visual features, before a residual MLP and classifier. This machinery lets the model emphasize, at each scale, the parts of the sticker that match the corresponding textual view.

What would settle it

Run the Table 4 view-order comparison with ten random seeds and report the spread: if the best order's average accuracy is within one standard deviation of the worst order's average, the hierarchical assignment to stages is not carrying the result. A second check is to retrain MGHFT with descriptions from a captioner that has no emotion knowledge; if accuracy stays near 73%, the MLLM's semantic content is not the active ingredient.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that sticker emotion classification improves when a Multimodal Large Language Model rewrites the sticker into four separate views (intention, overall style, main roles, and character details) and those views are fused into visual features hierarchically rather than all at once. Each view is paired with one of the four stages of a Pyramid Vision Transformer: soft attention enriches local visual tokens with the view text, while a contrastive loss plus a multi-level cross-entropy loss aligns the global CLS token with the same view. A text-guided fusion attention then re-integrates all stages. The paper reports that every component contributes, that concatenating all views at every stage hurts performance, and that the best configuration obtains 73.31% accuracy and 72.52% F1 on SER30K.

Load-bearing premise

The paper's largest assumption is that the specific ordering of the four text views across the four visual stages is what produces the improvement; the order is chosen after comparing alternatives, and several other orders score within about 0.3 accuracy points, so part of the gain may come from validation-set selection rather than from a genuine match to human perception.

Editorial extensions

If this is right

  • If the reported gains hold, sticker emotion recognition no longer needs OCR text or theme labels: generated multi-view descriptions can supply the missing context.
  • The hierarchical assignment of views to stages beats concatenating all descriptions at every stage, since the all-views-at-every-stage variant drops both accuracy and F1.
  • The gains over BLIP2 and CLIP indicate that a small trainable PVT with text guidance can outperform much larger frozen pre-trained models on this task.
  • The MET-MEME result suggests the approach transfers to low-resource and metaphor-rich sticker sets, though with a much smaller margin.
  • Each module - contrastive alignment, global fusion, local fusion, and text-guided attention - contributes, and removing contrastive alignment costs the most.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the paper's human-perception story for the order $T_1,T_2,T_3,T_4$ is not established by Table 4, since several alternative orders land within roughly 0.3 accuracy points; the order may be a mild validation-set selection rather than a perceptual necessity.
  • Inference: a natural stress test is to replace LLaVA-NeXT descriptions with a weaker captioner or with perturbed descriptions; if gains persist, the value comes from the fusion structure rather than the specific MLLM knowledge.
  • Inference: the same four-view recipe could be tried on meme sentiment or sarcasm detection, where implicit intent and style cues are similarly central; the paper does not test this.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces MGHFT, a multi-granularity hierarchical fusion transformer for sticker emotion recognition. The method uses LLaVA-NeXT to generate four-view textual descriptions of a sticker (intention, overall style, main roles, character details), encodes them with a frozen BERT model, and injects them stage-by-stage into a PVT visual backbone through a local Soft-Fusion attention mechanism, a contrastive/MLCE alignment loss at the global granularity, and a final Text-Guided Fusion Attention module. The authors report state-of-the-art results on SER30K (73.31% accuracy, 72.52% F1) and MET-MEME (35.13% accuracy), with component ablations, text-order experiments, and t-SNE/attention visualizations. The code is released.

Significance. The idea of using MLLM-generated multi-view descriptions as auxiliary text for sticker emotion recognition is reasonable, and the hierarchical stage-wise injection into a pyramid vision transformer is a plausible architectural contribution. The released code supports reproducibility. However, the central empirical claim is not yet isolated from a major confound: MGHFT consumes LLaVA-NeXT descriptions, while the Image+Text baselines consume OCR-derived text, so the reported gains may reflect the richer text source rather than the proposed fusion mechanism. In addition, all results are single-run with no error bars or significance tests. If the matched experiments confirm the advantage, this would be a useful contribution; at present the evidence is conditional.

major comments (3)
  1. [Section 4.3, Table 1 and Figure 1] The headline comparison confounds the proposed architecture with the text modality. The Image+Text baselines (LORA, TGCA-PVT) use their original OCR-based text pipelines, whereas MGHFT uses four LLaVA-NeXT-generated multi-view descriptions (Eqs. 1-2) encoded by frozen BERT. Because the LLaVA-NeXT descriptions provide a substantially richer external-knowledge signal, the reported 2.3% accuracy and 2.2% F1 gains over TGCA-PVT cannot be attributed to the hierarchical fusion mechanism. Please add matched baselines that receive the same LLaVA-NeXT descriptions with simpler fusion strategies (e.g., late concatenation, TGCA-PVT with LLaVA text, or MGHFT with OCR text) to isolate the architectural contribution.
  2. [Section 4.3, Tables 1-2] All experimental results are single runs with no error bars, confidence intervals, or significance tests. The reported advantages are small (2.3 accuracy points on SER30K and 0.77 points on MET-MEME), and Table 4 shows several configurations within 0.3-0.4 points of each other. Please report mean and standard deviation over at least three random seeds and perform a statistical test (e.g., paired bootstrap or McNemar) to support the word "significantly" used in the abstract and Section 4.3.
  3. [Section 4.5, Table 4] The paper claims that the [T1,T2,T3,T4] view-to-stage assignment mirrors human perception, but Table 4 shows that this order was selected empirically after testing permutations; differences among the tested orders are at most 0.34 accuracy points, and the single-view configuration T4 at every stage already reaches 72.92% accuracy versus 73.31% for the full model. These small margins do not strongly support the hierarchical-order rationale. Please provide variance estimates for these comparisons and either a principled justification for the order (e.g., a perception study or pre-registered hypothesis) or soften the claim accordingly.
minor comments (6)
  1. [Equation (5)] The definition of C_t uses f_l, but f_l is not defined anywhere in the paper; the intended quantity appears to be f_t. Please correct the notation.
  2. [Equation (3)] The text surrounding Eq. (3) refers to V'_t and T'_i inconsistently; the fused local feature should be defined without reusing undefined or ambiguous variables.
  3. [Equation (7)] Equation (7) sets h'_g equal to itself plus a soft-fusion term; please use distinct symbols for the input and output of the fusion step to avoid a recursive definition.
  4. [Section 4.3, MET-MEME results paragraph] The text states that MGMCF achieves "a precision of 34.36%, a precision of 37.77% and a recall of 34.88%" which is a typo; the values in Table 2 should be described consistently (accuracy, precision, recall).
  5. [Figure 1] The labels in Figure 1 use parenthesized pairs; please clarify that they denote (accuracy, F1) and specify how the percentage gains are computed.
  6. [General] The ACM template footer still contains the placeholder "Conference'17"; please update the venue and year information.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MGHFT is an empirically evaluated architecture whose gains are not derived from fitted inputs or a self-citation chain.

full rationale

The paper's derivation chain is empirical rather than definitional. The multi-view descriptions are generated by a frozen MLLM from the sticker image (Eq. 1), then encoded by a frozen BERT (Eq. 2); no emotion label is fed into the description-generation process, so the text is an additional input feature rather than a fitted target renamed as a prediction. The classification objective L_CE and the alignment loss (Eqs. 4-6, 9) are standard training losses, and the reported improvements on SER30K and MET-MEME (Tables 1-2) are benchmark comparisons, not consequences of an equation-level identity. The only self-citation is the authors' prior TGCA-PVT [5], which is used as a baseline to outperform rather than as load-bearing justification for the model's design; the loss coefficient lambda=30 is adopted from the external Ref. [45], and the backbone and text encoder are independently pretrained models. The Table 4 ordering of multi-view descriptions was selected empirically by permutation testing, which is a validation-set selection concern rather than circularity, and the comparison with BLIP2 is not input-matched because MGHFT consumes LLaVA-NeXT-generated descriptions while baselines do not; however, this is an experimental fairness issue, not a circular-derivation issue. No equation or cited prior result reduces the central claim to its own inputs, so the paper is self-contained against external benchmarks and receives a 0 circularity score.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical or external entities are introduced; the components (multi-view interpreter, soft-fusion, TGFA) are architectural modules implemented with existing networks. The free parameters are the empirically chosen view order and loss weights, which are not derived from first principles.

free parameters (4)
  • view-to-stage mapping order = [T1, T2, T3, T4]
    Selected after observing validation accuracy across permutations (Table 4); differences between orders are small (72.97-73.31), so the mapping is not uniquely determined.
  • MLCE loss weight lambda = 30
    Adopted from Ref. [45] without sensitivity analysis on this task (Eq. 6).
  • alignment loss weight = 0.5
    Hand-set in Eq. (9); no ablation on this coefficient.
  • temperature tau in alignment loss = not reported
    In Eq. (4) and Eq. (5); value not stated in the paper.
assumptions (4)
  • ad hoc to paper The four views (intention, overall style, main roles, character details) are sufficient to capture emotion-relevant sticker semantics.
    No evidence is provided that these four views are complete or minimal; Section 3.2.
  • domain assumption LLaVA-NeXT-generated descriptions are semantically reliable and complementary to the visual signal.
    The model's performance depends on this; no error analysis of the generated descriptions is given (Section 3.2).
  • domain assumption PVT's CLS tokens and attention-selected key tokens represent global and local sticker features respectively.
    Taken from prior work [5, 26, 39]; used as the backbone assumption in Section 3.1.
  • domain assumption The contrastive and MLCE losses improve cross-modal alignment for this task.
    Borrowed from Ref. [45]; no isolation of the alignment loss contribution beyond the CL ablation (Section 3.3.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of MGHFT: Multi-Granularity Hierarchical Fusion Transformer for Cross-Modal Sticker Emotion Recognition." pith.science (2026). https://pith.science/paper/V37PAKAH

@misc{pith2026250718929,
  author       = {Pith},
  title        = {Pith review of: MGHFT: Multi-Granularity Hierarchical Fusion Transformer for Cross-Modal Sticker Emotion Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V37PAKAH}},
  note         = {Machine review of arXiv:2507.18929}
}
read the original abstract

Although pre-trained visual models with text have demonstrated strong capabilities in visual feature extraction, sticker emotion understanding remains challenging due to its reliance on multi-view information, such as background knowledge and stylistic cues. To address this, we propose a novel multi-granularity hierarchical fusion transformer (MGHFT), with a multi-view sticker interpreter based on Multimodal Large Language Models. Specifically, inspired by the human ability to interpret sticker emotions from multiple views, we first use Multimodal Large Language Models to interpret stickers by providing rich textual context via multi-view descriptions. Then, we design a hierarchical fusion strategy to fuse the textual context into visual understanding, which builds upon a pyramid visual transformer to extract both global and local sticker features at multiple stages. Through contrastive learning and attention mechanisms, textual features are injected at different stages of the visual backbone, enhancing the fusion of global- and local-granularity visual semantics with textual guidance. Finally, we introduce a text-guided fusion attention mechanism to effectively integrate the overall multimodal features, enhancing semantic understanding. Extensive experiments on 2 public sticker emotion datasets demonstrate that MGHFT significantly outperforms existing sticker emotion recognition approaches, achieving higher accuracy and more fine-grained emotion recognition. Compared to the best pre-trained visual models, our MGHFT also obtains an obvious improvement, 5.4% on F1 and 4.0% on accuracy. The code is released at https://github.com/cccccj-03/MGHFT_ACMMM2025.

Figures

Figures reproduced from arXiv: 2507.18929 by the authors.

Figure 1
Figure 1. Comparison with pre-trained models on SER30K [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Framework of our proposed MGHFT. MGHFT adopts PVT as the backbone to extract the multi-granularity features of [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Multi-view sticker description generation. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Framework of TGFA for cross-modal fusion. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Performance of each model variants. helps align multi-view textual features with global visual represen￾tations, effectively narrowing the semantic gap between modalities and enhancing the performance of both GF and TGFA modules. Interestingly, the combination of GF an…
Figure 7
Figure 7. Figure 7: Visualization of the region of interest of the model [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 41 canonical work pages

  1. [1]

    Irving Biederman. 1987. Recognition-by-components: a theory of human image understanding. Psychological review 94, 2 (1987), 115

    work page 1987

  2. [2]

    Rui Cao, Roy Ka-Wei Lee, and Jing Jiang. 2024. Modularized networks for few- shot hateful meme detection. In Proceedings of the ACM Web Conference 2024 . 4575–4584

    work page 2024

  3. [3]

    Yoonjeong Cha, Jongwon Kim, Sangkeun Park, Mun Yong Yi, and Uichin Lee

    work page

  4. [4]

    Jiali Chen, Yi Cai, Ruohang Xu, Jiexin Wang, Jiayuan Xie, and Qing Li. 2024. Deconfounded Emotion Guidance Sticker Selection with Causal Inference. In Proceedings of the 32nd ACM International Conference on Multimedia . 3084–3093

    work page 2024

  5. [5]

    Jian Chen, Wei Wang, Yuzhu Hu, Junxin Chen, Han Liu, and Xiping Hu. 2024. TGCA-PVT: Topic-Guided Context-Aware Pyramid Vision Transformer for Sticker Emotion Recognition. In Proceedings of the 32nd ACM International Con- ference on Multimedia. 9709–9718

    work page 2024

  6. [6]

    Daantje Derks, Agneta H Fischer, and Arjan ER Bos. 2008. The role of emotion in computer-mediated communication: A review. Computers in human behavior 24, 3 (2008), 766–785

    work page 2008

  7. [7]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. CoRR abs/1810.04805 (2018). arXiv:1810.04805 http://arxiv.org/abs/1810.04805

    arXiv 2018

  8. [8]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

    arXiv 2020

Show all 60 references
  1. [9]

    Baishan Duan and Yuesheng Zhu. 2022. BROWALLIA at Memotion 2.0 2022: Multimodal memotion analysis with modified ogb strategies. In Proceedings of De-Factify: Workshop on Multimodal Fact Checking and Hate Speech Detection, CEUR

    2022

  2. [10]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition . 770–778

    2016

  3. [11]

    Byeongho Heo, Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Junsuk Choe, and Seong Joon Oh. 2021. Rethinking spatial dimensions of vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision . 11936– 11945

    2021

  4. [12]

    Nice picture comment!

    Susan Herring and Ashley Dainas. 2017. “Nice picture comment!” Graphicons in Facebook comment threads. (2017)

    2017

  5. [13]

    Yuxuan Hu, Minghuan Tan, Chenwei Zhang, Zixuan Li, Xiaodan Liang, Min Yang, Chengming Li, and Xiping Hu. 2024. Aptness: Incorporating appraisal theory and emotion support strategies for empathetic response generation. In Proceedings of the 33rd ACM International Conference on ...

    2024

  6. [14]

    Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger

  7. [15]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276 (2024)

    2024 arXiv

  8. [16]

    Prince Jha, Raghav Jain, Konika Mandal, Aman Chadha, Sriparna Saha, and Pushpak Bhattacharyya. 2024. Memeguard: An llm and vlm-based frame- work for advancing content moderation via meme intervention. arXiv preprint arXiv:2406.05344 (2024)

    2024 arXiv

  9. [17]

    Asad Khattak, Muhammad Zubair Asghar, Mushtaq Ali, and Ulfat Batool. 2022. An efficient deep learning technique for facial emotion recognition. Multimedia Tools and Applications 81, 2 (2022), 1649–1683

    2022

  10. [18]

    Byoung Chul Ko. 2018. A brief review of facial emotion recognition based on visual information. sensors 18, 2 (2018), 401

    2018

  11. [19]

    Gunther Kress and Theo Van Leeuwen. 2020. Reading images: The grammar of visual design. Routledge

    2020

  12. [20]

    Gitanjali Kumari, Kirtan Jain, and Asif Ekbal. 2024. M3Hop-CoT: Misogy- nous Meme Identification with Multimodal Multi-hop Chain-of-Thought. arXiv preprint arXiv:2410.09220 (2024)

    2024 arXiv

  13. [21]

    Joon Young Lee, Nahi Hong, Soomin Kim, Jonghwan Oh, and Joonhwan Lee. 2016. Smiley face: why we use emoticon stickers in mobile messaging. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (Florence, Ital...

    2016

  14. [22]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning . PMLR, 19730–19742

    2023

  15. [23]

    Bin Liang, Bingbing Wang, Zhixin Bai, Qiwei Lang, Mingwei Sun, Kaiheng Hou, Lanjun Zhou, Ruifeng Xu, and Kam-Fai Wong. 2024. Reply with Sticker: New Dataset and Model for Sticker Retrieval. arXiv preprint arXiv:2403.05427 (2024)

    2024

  16. [24]

    Hongzhan Lin, Ziyang Luo, Wei Gao, Jing Ma, Bo Wang, and Ruichao Yang

  17. [25]

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024. LLaVA-NeXT: Improved reasoning, OCR, and world knowl- edge. https://llava-vl.github.io/blog/2024-01-30-llava-next/

    2024

  18. [26]

    Shengzhe Liu, Xin Zhang, and Jufeng Yang. 2022. SER30K: A large-scale dataset for sticker emotion recognition. In Proceedings of the 30th ACM International Conference on Multimedia. 33–41

    2022

  19. [27]

    Min Luo, Boda Lin, Binghao Tang, Haolong Yan, and Si Li. 2024. ELEMO: Elements Focused Emotion Recognition for Sticker Images. InChinese Conference on Pattern Recognition and Computer Vision (PRCV) . Springer, 231–245

    2024

  20. [28]

    Khoi Nguyen and Vincent Ng. 2024. Computational Meme Understanding: A Survey. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 21251–21267

    2024

  21. [29]

    Jeongsik Park, Khoi PN Nguyen, Terrence Li, Suyesh Shrestha, Megan Kim Vu, Jerry Yining Wang, and Vincent Ng. 2024. MemeIntent: Benchmarking Intent Description Generation for Memes. In Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialog...

    2024

  22. [30]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  23. [31]

    Shivam Sharma, Siddhant Agarwal, Tharun Suresh, Preslav Nakov, Md Shad Akhtar, and Tanmoy Chakraborty. 2023. What do you meme? generating expla- nations for visual semantic role labelling in memes. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 9763–9771

    2023

  24. [32]

    Limor Shifman. 2013. Memes in digital culture . MIT press

    2013

  25. [33]

    Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

    2014 arXiv

  26. [34]

    Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning . PMLR, 6105–6114

    2019

  27. [35]

    Ying Tang and Khe Foon Hew. 2019. Emoticon, emoji, and sticker use in computer- mediated communication: A review of theories and research findings. Interna- tional journal of communication 13 (2019), 2457–2483

    2019

  28. [36]

    Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008)

    2008

  29. [37]

    Bingbing Wang, Shijue Huang, Bin Liang, Geng Tu, Min Yang, and Ruifeng Xu. 2024. What do they “meme”? A metaphor-aware multi-modal multi-task framework for fine-grained meme understanding. Knowledge-Based Systems 294 (2024), 111778

    2024

  30. [38]

    Shaojung Sharon Wang. 2016. More than words? The effect of line character sticker use on intimacy in the mobile communication environment.Social Science Computer Review 34, 4 (2016), 456–478

    2016

  31. [39]

    Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. 2021. Pyramid vision transformer: A versatile back- bone for dense prediction without convolutions. In Proceedings of the IEEE/CVF international conference on computer vi...

    2021

  32. [40]

    Wuyou Xia, Shengzhe Liu, Qin Rong, Guoli Jia, Eunil Park, and Jufeng Yang

  33. [41]

    Bo Xu, Tingting Li, Junzhe Zheng, Mehdi Naseriparsa, Zhehuan Zhao, Hongfei Lin, and Feng Xia. 2022. Met-meme: A multimodal meme dataset rich in metaphors. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval . 2887–2899

    2022

  34. [42]

    Bo Xu, Junzhe Zheng, Jiayuan He, Yuxuan Sun, Hongfei Lin, Liang Zhao, and Feng Xia. 2024. Generating Multimodal Metaphorical Features for Meme Under- standing. In Proceedings of the 32nd ACM International Conference on Multimedia . 447–455

    2024

  35. [43]

    Jingyuan Yang, Qirui Huang, Tingting Ding, Dani Lischinski, Danny Cohen-Or, and Hui Huang. 2023. Emoset: A large-scale visual emotion dataset with rich attributes. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 20383–20394

    2023

  36. [44]

    In Proceedings of the 32nd ACM International Conference on Multimedia

    Perceive before Respond: Improving Sticker Response Selection by Emo- tion Distillation and Hard Mining. In Proceedings of the 32nd ACM International Conference on Multimedia. 9631–9640

  37. [45]

    Rui Yang, Shuang Wang, Jianwei Tao, Yingping Han, Qiaoling Lin, YanHe Guo, Biao Hou, and Licheng Jiao. 2024. Accurate and Lightweight Learning for Spe- cific Domain Image-Text Retrieval. In Proceedings of the 32nd ACM International Conference on Multimedia. 9719–9728

    2024

  38. [46]

    Quanzeng You, Jiebo Luo, Hailin Jin, and Jianchao Yang. 2016. Building a large scale dataset for image emotion recognition: The fine print and the benchmark. In Proceedings of the AAAI conference on artificial intelligence , Vol. 30. Conference’17, July 2017, Washington, DC, U...

    2016

  39. [47]

    Chenwei Zhang, Yuxuan Hu, Min Yang, Chengming Li, and Xiping Hu. 2023. Skeletal Spatial-Temporal Semantics Guided Homogeneous-Heterogeneous Mul- timodal Network for Action Recognition. In Proceedings of the 31st ACM Interna- tional Conference on Multimedia . 3657–3666

    2023

  40. [48]

    Jufeng Yang, Dongyu She, Yu-Kun Lai, Paul L Rosin, and Ming-Hsuan Yang. 2018. Weakly supervised coupled networks for visual sentiment analysis. InProceedings of the IEEE conference on computer vision and pattern recognition . 7584–7592

    2018

  41. [49]

    Yiqun Zhang, Fanheng Kong, Peidong Wang, Shuang Sun, Lingshuai Wang, Shi Feng, Daling Wang, Yifei Zhang, and Kaisong Song. 2024. Stickerconv: generating multimodal empathetic responses from scratch. arXiv preprint arXiv:2402.01679 (2024)

    2024 arXiv

  42. [50]

    Sicheng Zhao, Zizhou Jia, Hui Chen, Leida Li, Guiguang Ding, and Kurt Keutzer

  43. [51]

    Sicheng Zhao, Xingxu Yao, Jufeng Yang, Guoli Jia, Guiguang Ding, Tat-Seng Chua, Bjoern W Schuller, and Kurt Keutzer. 2021. Affective image content analysis: Two decades review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 10 (2021), 6729–6751

    2021

  44. [52]

    Hao Zhang, Gaifang Luo, Yingying Yue, Kangjian He, and Dan Xu. 2024. Affective image recognition with multi-attribute knowledge in deep neural networks. Multimedia Tools and Applications 83, 6 (2024), 18353–18379

    2024

  45. [53]

    Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba

  46. [54]

    Yan Zhuang and Yanru Zhang. 2022. Yet at Memotion 2.0 2022: Hate speech detection combining bilstm and fully connected layers. In Proceedings of De- Factify: Workshop on Multimodal Fact Checking and Hate Speech Detection, CEUR

    2022

  47. [57]

    Li Zheng, Hao Fei, Ting Dai, Zuquan Peng, Fei Li, Huisheng Ma, Chong Teng, and Donghong Ji. 2025. Multi-Granular Multimodal Clue Fusion for Meme Understanding. arXiv preprint arXiv:2503.12560 (2025)

    2025 arXiv

  48. [59]

    Places: A 10 million image database for scene recognition.IEEE transactions on pattern analysis and machine intelligence 40, 6 (2017), 1452–1464

    2017

  49. [2017]

    In Proceedings of the IEEE conference on computer vision and pattern recognition

    Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition . 4700–4708

  50. [2018]

    Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 1–22

    Complex and ambiguous: Understanding sticker misinterpretations in instant messaging. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 1–22

    2018

  51. [2019]

    In Proceedings of the 27th ACM international conference on multimedia

    PDANet: Polarity-consistent deep attention network for fine-grained visual emotion regression. In Proceedings of the 27th ACM international conference on multimedia. 192–201

  52. [2024]

    In Proceedings of the ACM Web Conference 2024

    Towards explainable harmful meme detection through multimodal debate between large language models. In Proceedings of the ACM Web Conference 2024 . 2359–2370

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.