Pith. sign in

REVIEW 5 major objections 6 minor 71 references

Figurative-cum-Commonsense Knowledge Infusion for Multimodal Mental Health Meme Classification

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Injecting LLM-generated figurative-commonsense reasoning plus retrieved similar meme examples into a BART classifier is claimed to beat all baselines and the prior state of the art on fine-grained anxiety and depression meme classification.

desk verdict Useful new anxiety-meme dataset and a plausible pipeline, but the RAG evaluation has a label-leakage risk that needs to be closed before the F1 gains can be trusted. read the letter →

arxiv 2501.15321 v1 pith:TK4T5FHW submitted 2025-01-25 cs.CL cs.SI

Abdullah Mazhar , Zuhair hasan shaik , Aseem Srivastava , Polly Ruhnke , Lavanya Vaddavalli , Sri Keshav Katragadda , Shweta Yadav , Md Shad Akhtar This is my paper · ORCID
classification cs.CLcs.SI
keywords mentalhealthmemesfigurativelanguageunderstandingcommonsensereasoninganxietysymptomclassificationdepressionretrieval-augmentedgenerationAxiOMdatasetmultimodal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that fine-grained mental-health symptom labels can be read out of memes if the classifier is first given explicit figurative-commonsense knowledge, not just pixels and OCR text. To test this, the authors built AxiOM, a dataset of 3,582 memes labeled into six anxiety symptoms derived from the GAD questionnaire, and they propose M3H, which uses an LLM to generate structured reasoning about cause-effect, figurative understanding, and mental state for each meme, retrieves similar labeled examples from a retrieval-augmented database, and feeds all of this into a BART classifier. On AxiOM and on the public RESTORE depression dataset, M3H reports weighted-F1 gains of 4.20% and 4.66% over the prior state of the art, with larger macro-F1 gains. Ablation experiments show that removing the reasoning or retrieval modules hurts performance on both datasets. If the claim holds, it means explicit commonsense explanations are a practical way to improve machine understanding of figurative mental-health content.

What carries the argument

The pipeline has three stages. Stage one is figurative reasoning generation: an LLM is prompted to output three structured attributes for each meme—cause-effect relations, figurative understanding (metaphors, irony, symbols), and the depicted mental state. Stage two is knowledge fusion through retrieval-augmented generation: a sentence transformer creates embeddings of each training meme's OCR text and its reasoning, the two embeddings are concatenated, and for a new meme the top-n similar training instances are retrieved by cosine similarity. Stage three is classification: a BART encoder-decoder takes a prompt that includes the OCR text, the generated figurative reasoning, and the retrieved examples, and emits the symptom label, as a single softmax on AxiOM and per-label thresholded probabilities on RESTORE. The authors' argument is that the figurative reasoning supplies what the image-text pair by itself does not: the commonsense bridge between the meme's literal surface and its mental-health meaning.

What would settle it

Take the AxiOM test set and replace GPT-4o's figurative reasoning with human-authored gold explanations, and separately with randomly shuffled explanations drawn from other memes, keeping retrieval fixed. If the shuffled-reasoning condition retains the full 4.20% weighted-F1 gain over OCR-only BART, then the content of the reasoning is not what drives the result; if the human-reasoning condition raises scores further, the LLM explanations are the bottleneck.

Watch

Extended reading notes

Core claim

M3H's central claim is that combining LLM-generated figurative reasoning with retrieval-augmented exemplars and a strong text classifier gives the best reported results on mental-health meme symptom classification. On the new AxiOM anxiety dataset and the RESTORE depression dataset, the full M3H configuration reaches macro-F1 of 67.52 and 70.00 and weighted-F1 of 68.79 and 70.00, beating six baselines in 20 variations, including the prior state of the art, by 4.20% and 4.66% weighted-F1 respectively. The paper's ablation study shows that removing either the figurative-commonsense module or the retrieval module degrades performance on both datasets, which is the evidence that each component is load-bearing.

Load-bearing premise

The whole pipeline depends on GPT-4o's automatically written explanations being accurate and faithful to what each meme actually conveys; if those explanations are confidently wrong, the reported gains could shrink or reverse.

Editorial extensions

If this is right

  • If the claim holds, explicit LLM-generated commonsense reasoning is a portable ingredient that can be added to a text classifier to improve fine-grained mental-health classification without building a new multimodal architecture.
  • The AxiOM dataset fills a gap by providing a public benchmark for anxiety-meme classification with six GAD-based labels, where no dedicated public dataset existed before.
  • The retrieval module's contribution is separable and sizable: the reported ablation shows a jump of about 5 points when retrieval is added, so example-based knowledge infusion is doing real work beyond the reasoning text.
  • The error analysis identifies a concrete next target: confusion between 'Lack of Worry Control' and 'Impending Doom' shows that adjacent anxiety symptoms still need better disambiguation even when the model receives figurative reasoning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors do not test whether the content of the reasoning, rather than its mere presence, drives the gain; swapping the LLM's explanations for human-written ones or for deliberately vague ones would separate the content effect from the format effect.
  • Because the classifier consumes only OCR text, reasoning, and retrieved text, the image enters only indirectly through the LLM's reasoning; a testable extension is to ablate the image from the reasoning prompt to quantify how much visual information is actually being used.
  • The same recipe of LLM reasoning plus retrieval plus a strong text classifier is likely transferable to other figurative domains such as political cartoons or sarcastic product reviews, but the paper does not claim that and it would need its own dataset and baselines.
  • The paper explicitly disclaims diagnostic use; a downstream possibility, not a paper claim, would be to add a verification step that checks generated reasoning against the meme's visual content, since the authors list faithfulness of generated reasoning as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces AxiOM, a new dataset of 3,582 memes annotated for six GAD-derived anxiety symptoms, and proposes M3H, a framework that uses GPT-4o to generate figurative-commonsense reasoning (cause-effect, figurative understanding, mental state), retrieves similar training examples via a RAG module, and classifies with a BART head. M3H is benchmarked against 20 baseline variations on AxiOM and on the RESTORE depression dataset, with reported weighted-F1 improvements of 4.20% and 4.66% over the prior SOTA (Yadav et al.). The paper also includes ablation studies, a human evaluation of reasoning quality, and error analyses.

Significance. If the central claim holds, this is a useful contribution to multimodal mental-health meme classification: it provides a new clinically grounded anxiety-meme dataset, demonstrates a concrete way to inject LLM-generated figurative reasoning, and reports improvements over a wide set of baselines, with code and data released. The human evaluation and public dataset are assets. However, the specific numerical claims are weakened by internal arithmetic and ablation inconsistencies, and the RAG-based evaluation does not yet rule out a label-copying shortcut, so the paper's central attribution of gains to commonsense infusion is not yet established.

major comments (5)
  1. [Abstract; Section 1; Table 2] The reported improvements contain arithmetic errors that contradict the paper's own table. In Table 2, the M3H minus SOTA deltas are listed as 4.94, 4.20, 5.79, 4.66 for RESTORE macro-F1, RESTORE weighted-F1, AxiOM macro-F1, AxiOM weighted-F1. But the same table gives RESTORE macro-F1 67.52 vs 63.58 (difference 3.94, not 4.94) and AxiOM weighted-F1 70.10 vs 65.34 (difference 4.76, not 4.66). The abstract and Section 1 also state these deltas in an order that does not match Table 2 (the abstract says '4.20% and 4.66% on weighted-F1' without naming datasets; Section 1 says '4.94% and 5.79% on macro-F1 ... across AxiOM and RESTORE, respectively', while Table 2 shows the opposite mapping). Please correct these numbers and ensure the abstract, introduction, and Table 2 are mutually consistent.
  2. [Section 5.2; Appendix C, Table 4] The ablation reported in Table 2 does not match the detailed ablation in Appendix Table 4. The row labeled '– (FCS+RAG)' in Table 2 lists AxiOM macro-F1 64.28 and weighted-F1 64.97, but the corresponding row in Table 4 (OCR + FCS + FCS-RAG, without OCR-RAG) lists AxiOM macro-F1 64.94 and weighted-F1 64.95. Similarly, Table 2's '– (OCR+RAG)' row lists AxiOM 62.54/62.53, while Table 4's OCR + OCR-RAG + FCS row (without FCS-RAG) lists 61.82/61.79. These discrepancies affect the paper's summary of which component contributes what. Please reconcile Table 2 with Table 4 and clarify the notation: '– (FCS+RAG)' and '– (OCR+RAG)' are ambiguous about which components are removed.
  3. [Section 4.2; Table 5] The RAG evaluation does not rule out a label-copying shortcut. In the M3H prompt (Table 5), each retrieved training example is presented with its gold label (e.g., 'Difficulty Relaxing'). Retrieval is performed by cosine similarity in the joint OCR-text + figurative-reasoning embedding space (Eqs. 1–2), and the generated figurative reasoning frequently names the mental state explicitly (e.g., 'Mental State: Anxiety'). A test meme whose reasoning mentions its true class will likely retrieve training memes of the same class, so the BART classifier can achieve high F1 by copying the majority retrieved label. The paper does not report retrieval precision@k, does not compare against random, label-flipped, or label-masked retrieved examples, and does not analyze near-duplicate images or template variants between train and test. Please add these controls; without them the reported 4.20%/4.66% weighted-F1 gains cannot be attributed to figurative-commonsense infusion rather than to label leakage through retrieval.
  4. [Section 5.4; Section 6] The faithfulness of GPT-4o's figurative reasoning is under-evaluated. The paper's only direct check is a human evaluation of 40 samples (Section 5.4) that rates reasoning quality on a Likert scale; it does not test whether the generated reasoning contains incorrect mental-state attributions that would mislead classification. The Discussion (Section 6) itself acknowledges that LLMs 'may produce responses misaligned with human reasoning or fail to capture meme-specific nuances'. Given that this reasoning is the core input to both the RAG database and the classifier prompt, please provide a correctness-oriented evaluation (e.g., human verdicts on whether the generated mental state matches the gold label) or an ablation using oracle, random, or corrupted reasoning to show that classification is robust to reasoning errors.
  5. [Section 5.2; Appendix C] There are additional internal inconsistencies in the results narrative. Section 5.2 states that M3H achieves 'F1 scores of 67.52 (+494%) and 68.79 (+4.20%), and 70.09' without saying these are RESTORE's macro, weighted, and micro F1; the '+494%' is presumably '+4.94%', which is itself incorrect (see above). The same paragraph says 'we observe a consistent improvement on RESTORE data as well, with 70.00% score on both macro and weighted-F1', but Table 2 shows 70.00 for AxiOM, not RESTORE. Appendix C reports 'ΔRAG = 5.40' while the main text says '+5.09%'. Please correct these statements and make the dataset-order conventions consistent throughout.
minor comments (6)
  1. [Throughout] The model name is inconsistently written as GPT-4o in most places but GPT-4 in Section 5.4, Section 7, and Figure 6; please use a single convention.
  2. [Table 2] The table header writes 'LLA V A' and 'OCR + LLA V A' with an unusual space; this appears to be a typesetting artifact that should be fixed to LLaVA.
  3. [Section 5.1; Appendix B] The notation for hyperparameters and thresholds is sometimes inconsistent: Appendix B says the learning rate scheduler was 'constant', but Table 3 lists 'constant' as the value; more importantly, the RESTORE multilabel thresholds τ_j (Eq. 3) are mentioned but their values or selection procedure are never described. Please state how τ_j were set.
  4. [Figure 7; Section 5.2] Figure 7 reports ΔRAG = 5.40% but Section 5.2 says '+5.09%'; please verify which number is correct and cite the figure consistently.
  5. [Section 5.4] The human evaluation uses only 40 samples (20 per dataset) and reports only the majority rating distribution; please provide inter-annotator agreement or a more detailed breakdown, and state whether the same GPT-4o-generated reasoning was used for both the RAG database and the human evaluation.
  6. [Section 4.2] Equation (1) uses a concatenation of embeddings to form R^{n×2d}, but the subsequent retrieval in Eq. (2) uses cosine similarity; concatenated embeddings are known to distort cosine geometry. Please justify this choice or compare with alternative fusion (e.g., addition or a learned projection).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the M3H pipeline is an empirical feature-generation and retrieval-augmented classification system, and its reported gains are measured against external baselines and ablations.

full rationale

The paper's derivation chain is not circular. M3H is an empirical pipeline: GPT-4o generates figurative reasoning from the meme image and OCR text; Eq. (1)-(2) build a retrieval index from training OCR and reasoning; and the BART classifier receives OCR, reasoning, and retrieved labeled examples. No fitted parameter is renamed as a prediction: the reported 4.20%/4.66% weighted-F1 gains are against six external baselines (including Yadav et al.), and the ablations in Table 2 and Table 4 remove each module separately. One could raise an evaluation-validity concern that the retrieved in-context examples carry gold labels and that GPT-4o's 'Mental State' attribute may lexically approximate the symptom class, so retrieval could act as a label shortcut. That is a legitimate threat to the interpretation of the RAG contribution, but it is not constructional circularity: the test label never enters the retrieval computation or the prompt, and the feature-generating model is not fitted to the test set. The paper's self-citations (e.g., MOMENTA and prior mental-health work) appear only in related work and are not load-bearing; no uniqueness theorem or prior result is invoked to force the framework's design. The human evaluation in Section 5.4 is a quality check, not a fitted parameter. Therefore no circular step can be exhibited from the paper's own equations or citations, and the appropriate score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the quality of the LLM-generated reasoning, the validity of the annotation guidelines, and the post-hoc decision to drop one RESTORE class. These are domain assumptions rather than formal axioms. The free parameters (thresholds, retrieval count, embedding model, LLM settings) are not reported, which limits exact reproduction but does not create mathematical circularity.

free parameters (4)
  • Multilabel thresholds τ_j (RESTORE) = not reported
    Eq. 3 defines per-class thresholds for the multilabel RESTORE classification; their values are not given and presumably tuned on validation.
  • Retrieval count top-n (M3H RAG) = not reported (figure suggests K=3)
    Section 4.2 retrieves the top-n most similar examples; n is not stated in the text or appendix.
  • Sentence-transformer model for embeddings = not specified
    Eq. 1 and 2 use a sentence-transformer Π without naming the model; retrieval quality depends on this choice.
  • GPT-4o model snapshot and decoding parameters = not specified
    Figurative reasoning is generated by GPT-4o (Section 4.1); the exact version, temperature, and other decoding settings are absent.
assumptions (3)
  • domain assumption GPT-4o-generated figurative reasoning is faithful and useful for symptom classification.
    M3H's retrieval and classifier are built on this reasoning; validated only on 40 samples in Section 5.4.
  • domain assumption The GAD-based annotation guidelines correctly map memes to the six anxiety symptoms.
    Dataset labels from six annotators with 81% IUA and healthcare professional review; assumed reliable for benchmarking.
  • domain assumption The RESTORE 'Lack of Energy' category annotations are inconsistent and should be excluded.
    Section 3.1 excludes the class based on the authors' observation, changing the task from 8 to 7 classes and affecting comparability with prior results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Figurative-cum-Commonsense Knowledge Infusion for Multimodal Mental Health Meme Classification." pith.science (2026). https://pith.science/paper/TK4T5FHW

@misc{pith2026250115321,
  author       = {Pith},
  title        = {Pith review of: Figurative-cum-Commonsense Knowledge Infusion for Multimodal Mental Health Meme Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TK4T5FHW}},
  note         = {Machine review of arXiv:2501.15321}
}
read the original abstract

The expression of mental health symptoms through non-traditional means, such as memes, has gained remarkable attention over the past few years, with users often highlighting their mental health struggles through figurative intricacies within memes. While humans rely on commonsense knowledge to interpret these complex expressions, current Multimodal Language Models (MLMs) struggle to capture these figurative aspects inherent in memes. To address this gap, we introduce a novel dataset, AxiOM, derived from the GAD anxiety questionnaire, which categorizes memes into six fine-grained anxiety symptoms. Next, we propose a commonsense and domain-enriched framework, M3H, to enhance MLMs' ability to interpret figurative language and commonsense knowledge. The overarching goal remains to first understand and then classify the mental health symptoms expressed in memes. We benchmark M3H against 6 competitive baselines (with 20 variations), demonstrating improvements in both quantitative and qualitative metrics, including a detailed human evaluation. We observe a clear improvement of 4.20% and 4.66% on weighted-F1 metric. To assess the generalizability, we perform extensive experiments on a public dataset, RESTORE, for depressive symptom identification, presenting an extensive ablation study that highlights the contribution of each module in both datasets. Our findings reveal limitations in existing models and the advantage of employing commonsense to enhance figurative understanding.

Figures

Figures reproduced from arXiv: 2501.15321 by the authors.

Figure 1
Figure 1. Our problem statement involves the classification of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Label-wise sample of memes in the proposed dataset, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Proposed Framework: M3H. The meme image and OCR text are standalone inputs to our framework. Primarily, we employ LLM for figurative reasoning on prominent commonsense attributes. We deploy the reasoning into a RAG-database to be considered for knowledge fusion during classification. Later, we employ an encoder-decoder framework for final classification. input with a few examples in the Appendix (c.f. Section D, [P… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Confusion matrix computed on our proposed dataset, [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Error Analysis. We attempt to connect the dots between misclassification and figurative reasoning. Evidently, the reasoning [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Human Evaluation for commonsense reasoning using [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: There is a significant increase in the score when RAG [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

71 extracted references · 56 canonical work pages

  1. [1]

    Umair Akram and Jennifer Drabble. 2022. Mental health memes: beneficial or aversive in relation to psychiatric symptoms? Humanities and Social Sciences Communications 9, 1 (2022), 1–6

    work page 2022

  2. [2]

    Ankita Bhaumik and Tomek Strzalkowski. 2024. Towards a Generative Approach for Emotion Detection and Reasoning. arXiv preprint arXiv:2408.04906 (2024)

    arXiv 2024

  3. [3]

    Sravani Boinepelli. 2022. Towards Identification, Classification and Analysis of Mental Illness on Social Media . Ph. D. Dissertation. International Institute of Information Technology, Hyderabad

    work page 2022

  4. [4]

    Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. COMET: Commonsense Transformers for Automatic Knowledge Graph Construction. arXiv:1906.05317 [cs.CL] https://arxiv.org/abs/1906.05317

    arXiv 2019

  5. [5]

    Luca Braghieri, Ro’ee Levy, and Alexey Makarin. 2022. Social media and mental health. American Economic Review 112, 11 (2022), 3660–3693

    work page 2022

  6. [6]

    Pankaj K Choudhary and HN Nagaraja. 2005. Assessment of agreement us- ing intersection-union principle. Biometrical Journal: Journal of Mathematical Methods in Biosciences 47, 5 (2005), 674–681

    work page 2005

  7. [7]

    Munmun De Choudhury. 2013. Role of social media in tackling challenges in mental health. InProceedings of the 2nd international workshop on Socially-aware multimedia. 49–52

    work page 2013

  8. [8]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805 [cs.CL] https://arxiv.org/abs/1810.04805

    arXiv 2019

Show all 71 references
  1. [9]

    Yuhao Du, Muhammad Aamir Masood, and Kenneth Joseph. 2020. Understanding visual memes: An empirical analysis of text superimposed on memes shared on twitter. In Proceedings of the International AAAI Conference on Web and Social Media, V ol. 14. 153–164

    2020

  2. [10]

    Myryll Esco, Kenneth Llanasas, Vida Zoe Salino, Carmel Guazon, and Lucy L Teves. 2023. Perceived Attitude of Facebook Users Towards Memes About Depression. (2023)

    2023

  3. [11]

    Aastha Gupta, Deepanshi Gupta, Pallavi Chaudhary, Pragya Raje, and Komal Chandiramani. 2021. Exploring trivialisation of mental health issues using internet memes in young adults. The (2021), 868–81

    2021

  4. [12]

    Ameer Hamza, Abdul Rehman Javed, Farkhund Iqbal, Amanullah Yasin, Gautam Srivastava, Dawid Połap, Thippa Reddy Gadekallu, and Zunera Jalil. 2024. Multi- modal religiously hateful social media memes classification based on textual and image data. ACM Transactions on Asian and L...

    2024

  5. [13]

    Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. DeBERTa: Decoding-enhanced BERT with Disentangled Attention. arXiv:2006.03654 [cs.CL] https://arxiv.org/abs/2006.03654

    2021 arXiv

  6. [14]

    Blake Anthony Hickey, Taryn Chalmers, Phillip Newton, Chin-Teng Lin, David Sibbritt, Craig S McLachlan, Roderick Clifton-Bligh, John Morley, and Sara Lal

  7. [15]

    Shaoxiong Ji, Tianlin Zhang, Luna Ansari, Jie Fu, Prayag Tiwari, and Erik Cam- bria. 2021. MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare. arXiv:2110.15621 [cs.CL] https://arxiv.org/abs/2110.15621

    2021 arXiv

  8. [16]

    Kimesha L Jones. 2022. Perceptions of Facebook by Individuals with Mental Health Vulnerabilities. Northcentral University

    2022

  9. [17]

    Saurav Joshi, Filip Ilievski, and Luca Luceri. 2024. Contextualizing Internet Memes Across Social Media Platforms. In Companion Proceedings of the ACM on Web Conference 2024. 1831–1840

    2024

  10. [18]

    Abdul Aziz Turhan Kariko and Nonny Anasih. 2019. Laughing at one’s self: A study of self-reflective internet memes. In Journal of physics: conference series, V ol. 1175. IOP Publishing, 012250

    2019

  11. [19]

    Naseer Ahmed Khan, Samer Abdulateef Waheeb, Atif Riaz, and Xuequn Shang

  12. [20]

    Alexander V Laskin and Yasha A Laskin. 2022. Memes: Interactive Creative Intertextuality on Social Media. The SAGE Handbook of Social Media Marketing (2022), 307

    2022

  13. [21]

    Biomolecules 11, 8 (2021), 1093

    A novel knowledge distillation-based feature selection for the classification of ADHD. Biomolecules 11, 8 (2021), 1093

    2021

  14. [22]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual Instruction Tuning. arXiv:2304.08485 [cs.CV] https://arxiv.org/abs/2304.08485

    2023 arXiv

  15. [23]

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. arXiv:1910.13461 [cs.CL] http...

    2019 arXiv

  16. [24]

    Ganeshan Malhotra, Abdul Waheed, Aseem Srivastava, Md Shad Akhtar, and Tanmoy Chakraborty. 2022. Speaker and Time-aware Joint Contextual Learn- ing for Dialogue-act Classification in Counselling Conversations (WSDM ’22). Association for Computing Machinery, New York, NY , USA,...

    2022

  17. [25]

    Usha Lokala, Aseem Srivastava, Triyasha Ghosh Dastidar, Tanmoy Chakraborty, Md Shad Akhtar, Maryam Panahiazar, and Amit Sheth. 2022. A Computational Approach to Understand Mental Health from Reddit: Knowledge-Aware Multitask Learning Framework. Proceedings of the International...

    2022 doi

  18. [26]

    Laura Manea, Simon Gilbody, and Dean McMillan. 2015. A diagnostic meta- analysis of the Patient Health Questionnaire-9 (PHQ-9) algorithm scoring method as a screen for depression. General hospital psychiatry 37, 1 (2015), 67–75

    2015

  19. [27]

    Suresh Malodia, Amandeep Dhir, Anil Bilgihan, Pranao Sinha, and Tanishka Tikoo. 2022. Meme marketing: How can marketers drive better engagement using viral memes? Psychology & Marketing 39, 9 (2022), 1775–1801

    2022

  20. [28]

    Saif M Mohammad. 2024. WorryWords: Norms of Anxiety Association for over 44k English Words. arXiv preprint arXiv:2411.03966 (2024)

    2024 arXiv

  21. [29]

    Elly Atiqah Md Nawawi, Muhammad Faiz Izzudin, Nur Atiqah Mohd Zulkafli, and Siti Sarah Januri. 2020. Identifying the Factors Affecting Internet Memes to Become Viral on Social Media. (2020)

    2020

  22. [30]

    OpenAI. 2024. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL] https: //arxiv.org/abs/2303.08774

    2024 arXiv

  23. [31]

    Jessica Gall Myrick, Robin L Nabi, and Nicholas J Eng. 2022. Consuming memes during the COVID pandemic: Effects of memes and meme type on COVID-related stress and coping efficacy. Psychology of Popular Media 11, 3 (2022), 316

    2022

  24. [32]

    Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty

    Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md. Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. 2021. MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets. In Findings of the Association for Computational Linguistics: EMNLP 2021, Marie-Fran...

    2021

  25. [33]

    Umera Wajeed Pasha. 2024. Multilingual sexism detection in memes, a CLIP- enhanced machine learning approach. Working Notes of CLEF (2024)

    2024

  26. [34]

    Adam G Sanford, Stacy L Smith, and Dinur Blum. 2023. Going viral: how social media increased the spread of COVID-19 misinformation. In COVID-19: Individual Rights and Community Responsibilities. Routledge, 54–76

    2023

  27. [35]

    Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. 2021. MOMENTA: A multimodal framework for detecting harmful memes and their targets. arXiv preprint arXiv:2109.05184 (2021)

    2021 arXiv

  28. [36]

    E. S. Smitha, S. Sendhilkumar, and G. S. Mahalaksmi. 2018. Meme Classification Using Textual and Visual Features. InComputational Vision and Bio Inspired Com- puting, D. Jude Hemanth and S. Smys (Eds.). Springer International Publishing, Cham, 1015–1031

    2018

  29. [37]

    Shivam Sharma, Firoj Alam, Md Shad Akhtar, Dimitar Dimitrov, Giovanni Da San Martino, Hamed Firooz, Alon Halevy, Fabrizio Silvestri, Preslav Nakov, and Tanmoy Chakraborty. 2022. Detecting and understanding harmful memes: A survey. arXiv preprint arXiv:2205.04274 (2022)

    2022 arXiv

  30. [38]

    Aseem Srivastava, Smriti Joshi, Tanmoy Chakraborty, and Md Shad Akhtar. 2024. Knowledge Planning in Large Language Models for Domain-Aligned Counseling Summarization. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, M...

    2024 doi

  31. [39]

    Aseem Srivastava, Tanya Gupta, Alison Cerezo, Sarah Peregrine (Grin) Lord, Md Shad Akhtar, and Tanmoy Chakraborty. 2025. Critical behavioral traits foster peer engagement in Online Mental Health Communities. PLOS ONE 20, 1 (01 2025), 1–17. https://doi.org/10.1371/journal.pone.0316906

    2025 doi

  32. [40]

    Viriya Taecharungroj and Pitchganut Nueangjamnong. 2014. The effect of humour on virality: The study of Internet memes on social media. In 7th International Forum on Public Relations and Advertising Media Impacts on Culture and Social Communication. Bangkok, August. 151–166

    2014

  33. [41]

    Shardul Suryawanshi and Bharathi Raja Chakravarthi. 2021. Findings of the Shared Task on Troll Meme Classification in Tamil. In Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages, Bharathi Raja Chakravarthi, Ruba Priyadharshini, Anand...

    2021

  34. [42]

    Fan Wu, Guolian Chen, Junkuo Cao, Yuhan Yan, and Zhongneng Li. 2024. Multimodal Hateful Meme Classification Based on Transfer Learning and a Cross-Mask Mechanism. Electronics 13, 14 (2024). https://doi.org/10.3390/ electronics13142780

    2024

  35. [43]

    Sorina Vasile, Lebogang Mototo, and Tinashe Chuchu. 2021. Using “Memes” as a marketing communication tool in connecting with consumers within the age of digital connectivity. International Review of Management and Marketing 11, 3 (2021), 30–35

    2021

  36. [44]

    Shweta Yadav, Cornelia Caragea, Chenye Zhao, Naincy Kumari, Marvin Solberg, and Tanmay Sharma. 2023. Towards identifying fine-grained depression symptoms 9 WWW ’25, April 28-May 2, 2025, Sydney, NSW, Australia Mazhar et al. from memes. In Proceedings of the 61st Annual Meeting...

    2023

  37. [45]

    Lexing Xie, Apostol Natsev, John R Kender, Matthew Hill, and John R Smith

  38. [46]

    Kailai Yang, Tianlin Zhang, Ziyan Kuang, Qianqian Xie, and Sophia Ananiadou

  39. [47]

    Yang Zhong and Bhiman Kumar Baghel. 2024. Multimodal Understanding of Memes with Fair Explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. 2007–2017

    2024

  40. [48]

    Shweta Yadav, Cornelia Caragea, Chenye Zhao, Naincy Kumari, Marvin Solberg, and Tanmay Sharma. 2023. Towards Identifying Fine-Grained Depression Symp- toms from Memes. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa...

    2023 doi

  41. [52]

    I DONT ALW AYS SLEEP AT NIGHT BUT WHEN I DO, I DONT

    Dawei Zhou, Jiebo Luo, Vincent Silenzio, Yun Zhou, Jile Hu, Glenn Currier, and Henry Kautz. 2015. Tackling mental health by integrating unobtrusive multimodal sensing. In Proceedings of the AAAI conference on artificial intelligence, V ol. 29. 10 Figurative-cum-Commonsense Kno...

    2015

  42. [53]

    Cause: Not sleeping regularly at night Effect: Feeling tired or anxious during the day

  43. [54]

    Figurative Understanding

    Cause: Drinking alcohol (implied with the beer bottle) Effect: Difficulty in sleeping or restlessness at night. Figurative Understanding

  44. [55]

    I don’t always sleep at night

    Metaphor: "I don’t always sleep at night" Interpretation: Suggests a chaotic or irregular sleep pattern that many can relate to, often due to anxiety

  45. [56]

    But when I do, I don’t

    Irony: "But when I do, I don’t" Interpretation: Implies that even when he does sleep, it’s not restful, highlighting the contradiction in seeking rest yet feeling unrested. Mental State

  46. [57]

    Mental State: Anxiety or restlessness at night Description: Reflects the inner turmoil of being unable to sleep due to worries or stress

  47. [58]

    am I exhausted? Yes will I go to bed earlier? No

    Mental State: Humor or sarcasm Description: The playful tone of the statement conveys a sense of humor about the struggles of insomnia or disrupted sleep patterns. <|Assistant|> The mental health disorder of the person for this post is: Difficulty Relaxing Example-2: <|ocr_tex...

  48. [59]

    Cause: The individual feels exhausted

  49. [60]

    Figurative Understanding:

    Effect: Despite this exhaustion, the individual will not go to bed earlier. Figurative Understanding:

  50. [61]

    Symbolic Meaning: The exhaustion represents a common struggle with prioritizing self-care

  51. [62]

    Mental State:

    Humorous Undertone: The light-hearted refusal to go to bed earlier despite acknowledgment of exhaustion illustrates a relatable, ironic attitude towards personal habits. Mental State:

  52. [63]

    Mental State: The individual is feeling exhausted

  53. [64]

    Dad: you’re up early

    Emotional State: There is a sense of defiance or acceptance of unhealthy habits, possibly mixed with humor or resignation. <|Assistant|> The mental health disorder of the person for this post is: Difficulty Relaxing Your turn: <|ocr_text|> "Dad: you’re up early" - "Me:" - "Dad...

  54. [65]

    Effect: The dad questions the child’s sleep pattern

    Cause: The child is up early. Effect: The dad questions the child’s sleep pattern

  55. [66]

    Effect: The dad realizes the child may be sleep-deprived or acting unusually

    Cause The child did not go to sleep. Effect: The dad realizes the child may be sleep-deprived or acting unusually. Figurative Understanding:

  56. [67]

    Figurative Meaning The dad’s smile symbolizes an awareness of common teenage behavior (e.g., staying up late)

  57. [68]

    Underlying Message: It humorously highlights a typical parent-child dynamic regarding sleep schedules

  58. [69]

    Mental State:

    Irony The child being up early might typically be seen as positive, but it signifies a lack of sleep instead, turning a seemingly good situation into a concerning one. Mental State:

  59. [70]

    Emotion: Curiosity mixed with a hint of concern

    Mental State of the Dad: Amusement or bemusement at the child’s situation. Emotion: Curiosity mixed with a hint of concern

  60. [71]

    Emotion: Discomfort or embarrassment due to the revelation of staying awake all night

    Mental State of the Child Possibly anxious or guilty about not sleeping. Emotion: Discomfort or embarrassment due to the revelation of staying awake all night. <|Assistant|> The mental health disorder of the person for this post is: || M3H Prediction: Difficulty Relaxing Table...

  61. [2011]

    In Proceedings of the 19th ACM international conference on Multimedia

    Visual memes in social media: tracking real-world news in youtube videos. In Proceedings of the 19th ACM international conference on Multimedia. 53–62

  62. [2021]

    Sensors 21, 10 (2021), 3461

    Smart devices and wearable technologies to detect and monitor mental health conditions and stress: A systematic review. Sensors 21, 10 (2021), 3461

    2021

  63. [2023]

    arXiv:2309.13567 [cs.CL]

    MentalLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Models. arXiv:2309.13567 [cs.CL]

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.