Pith. sign in

REVIEW 5 major objections 6 minor 55 references

On VLMs for Diverse Tasks in Multimodal Meme Classification

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that fine-tuning small text classifiers on VLM-generated meme explanations outperforms both direct VLM prompting and LoRA adapter tuning, with the largest gain on sentiment classification.

desk verdict A useful but under-controlled distillation recipe for meme classification; the sentiment gains are real, but the paper never isolates the visual contribution from the OCR text. read the letter →

arxiv 2505.20937 v1 pith:JNV4KZGG submitted 2025-05-27 cs.CL

Deepesh Gavit , Debajyoti Mazumder , Samiran Das , Jasabanta Patro This is my paper · ORCID
classification cs.CL
keywords memeclassificationvision-languagemodelschain-of-thoughtpromptingLoRAfine-tuningknowledgedistillationmultimodalunderstandingsarcasmdetectionsentimentanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether vision-language models (VLMs) can be put to work on meme classification, and argues that the most effective use is not to query them directly or fine-tune them with adapters, but to have them write detailed textual explanations of memes and then train small text-only classifiers on those explanations. This distillation pipeline, named CoVExFiL, beats both direct prompting and LoRA adapter tuning on the Memotion and MAMI benchmarks. The gains are biggest on sentiment classification, with roughly a 26 percent relative improvement over task-specific state-of-the-art models; sarcasm and offensiveness improve by smaller margins. The paper's own error analysis shows the method inherits the VLM's blind spots, so the ceiling is set by explanation quality.

What carries the argument

The carrying mechanism is CoVExFiL (Combining VLM Explanation to Fine-tune LLMs), a two-stage distillation from vision-language models to small language models, with the meme explanation as the transfer currency. In stage one, VLMs are prompted with zero-shot, few-shot, or chain-of-thought prompts to describe the image, examine the embedded text, and combine the two into a judgment; in stage two, a text-only classifier is fine-tuned on those explanations with the meme label. The three-step chain-of-thought prompt is the version that produces the most useful explanations.

What would settle it

Hold a test set of memes whose labels depend on a visual element not named in the embedded text, such as a recognizable person in the image that flips the meaning. If a classifier trained on VLM explanations still classifies these correctly when the explanations systematically omit that element, the proposed explanation-as-distillation mechanism is not the source of the gains. A cleaner test is to delete all visual references from the VLM explanations and re-run; if accuracy does not drop, the explanations are not carrying the visual information the paper claims they carry.

Watch

Extended reading notes

Core claim

The central discovery is that the bottleneck in meme classification is not the classifier's size or the label signal, but whether the visual-contextual reasoning of a VLM can be captured in language and transferred. By prompting VLMs to produce structured, step-by-step interpretations of a meme and then fine-tuning small language models on those texts, the paper obtains weighted-F1 scores that exceed direct VLM prompting (46.12 percent average versus 41.48 percent) and LoRA-based VLM fine-tuning (36.89 percent). The best version uses chain-of-thought prompts, and higher BERTScore agreement with reference explanations tracks higher classification accuracy.

Load-bearing premise

The method assumes that whatever a vision-language model writes about a meme preserves the exact visual hint that determines the correct label; if that hint is left out, the text-only classifier cannot recover it.

Editorial extensions

If this is right

  • CoVExFiL makes meme classification cheaper at inference time, because a small text classifier substitutes for a large vision-language model after training.
  • Chain-of-thought prompting should be preferred when generating training explanations, because structured reasoning yields better downstream classifiers.
  • LoRA fine-tuning of VLMs is not a reliable route to meme understanding, at least with the few layers and epochs tested.
  • Explanation quality metrics such as BERTScore can be used to anticipate which memes a distilled classifier will get wrong.
  • Sarcasm and offensiveness remain the hard core of meme understanding; gains there are real but small.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If explanation fidelity is the ceiling, then filtering or re-prompting low-quality VLM explanations before training could push the method past the numbers reported here.
  • The same distillation recipe could be transferred to other multimodal judgment tasks where the decisive evidence is visual and contextual, such as hate-speech detection in images or propaganda analysis.
  • Because the training signal is text distilled by a VLM, any cultural or annotator bias in the explanation will be inherited and amplified by the downstream classifier; using multiple VLMs and keeping only agreed-upon explanations might mitigate that.
  • A testable extension suggested by the paper's BERTScore analysis is to use explanation quality as a per-sample confidence score, routing easy memes to the cheap classifier and hard ones to human review.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper presents a systematic study of vision-language models (VLMs) for meme classification on the Memotion and MAMI datasets. It benchmarks four VLMs (LLaVA-1.6, Qwen2-VL, LLaMA-3.2-Vision, InstructBLIP) under zero-shot, zero-shot chain-of-thought, few-shot, and few-shot chain-of-thought prompting, and under LoRA fine-tuning. It then proposes CoVExFiL, which generates VLM-based textual explanations of memes and fine-tunes small text-only LLMs (BERT, RoBERTa, XLNet) on those explanations for classification. The paper reports that CoVExFiL outperforms both direct VLM prompting and LoRA tuning, with a 26.14% relative improvement over the state of the art for sentiment classification and smaller gains for sarcasm and offensiveness. The main claim is that VLM explanation-based fine-tuning provides a viable distillation strategy for multimodal meme understanding.

Significance. If the central claim holds, CoVExFiL offers a practical and cost-effective alternative to fine-tuning large VLMs: a small text classifier can inherit multimodal understanding through VLM-generated explanations. The paper's breadth—four VLMs, multiple prompting strategies, adapter tuning, and two datasets—is a useful empirical contribution, and the error analysis in Appendix C identifies concrete failure modes (e.g., dropped visual cues, over-reliance on surface features) that are valuable for future work. However, the significance is tempered by the lack of a critical text-only control, the apparent selection of configurations on the test sets, and the absence of statistical rigor and reproducibility details. These issues must be addressed before the central claim can be considered established.

major comments (5)
  1. [§3.3, Fig. 5, §4.3] The central claim that CoVExFiL outperforms direct prompting and LoRA tuning is attributed to the VLM's multimodal understanding, but no experiment isolates the contribution of the visual/contextual channel. The VLM explanations used to fine-tune the LLMs transcribe the meme's embedded text; the examples in Fig. 5 quote on-image captions such as "WOMAN, GET OVER HERE THIS INSTANT." A control that trains the same LLMs on OCR text alone, or on meme captions with the visual analysis removed, is necessary to determine whether the reported gains come from visual semantics or merely from learning a stronger text-only classifier. Without such a control, the mechanism behind the 26.14% SN improvement is not established. The error analysis in C.3, which shows that dropping a decisive visual cue (the Nazi rally image) leads to misclassification, supports the existence of an explanation-quality ceiling but does not replace the missing text-only baseline. This experiment is straightforward to add and is essential to the paper's main claim.
  2. [§3.3, Tables 1–3] The paper reports the highest F1 among many (VLM, prompt, LLM) combinations without describing any validation procedure. Table 3 contains 21 columns and 12 rows, and the text selects the maximum per task and model; there is no separate validation split, no repeated evaluation, and no statement about how many configuration choices were made after inspecting the test results. This selection on the test data overstates the expected performance and makes the headline comparison to SOTA unreliable. Please specify a validation strategy (e.g., a fixed dev split or cross-validation), report the validation-based selection, and then give the test scores of the selected configurations.
  3. [§4.3, Table 7] The claimed 26.14% relative improvement over SOTA for SN, and the smaller gains for SR and OF, are computed against published numbers from prior work (Table 7), but the paper does not establish that those numbers were obtained under the same train/test splits, preprocessing, and evaluation metric. For example, M2Seq2Seq-MLD is reported on Memotion by its authors, and its F1 may be computed with a different protocol (e.g., macro vs. weighted). To support a SOTA claim, re-run these baselines under the same protocol or clearly qualify the comparison as a reference point rather than a strict improvement. Without this, the headline improvement is not verifiable.
  4. [§3.3, Appendix B.2.1] For the proposed CoVExFiL method, the paper provides no hyperparameters for fine-tuning the LLMs: learning rate, number of epochs, batch size, optimizer, and random seeds are absent, and no code is released. Table 8 only covers LoRA hyperparameters for VLMs, not the LLM fine-tuning in Exp 3. As a result, the central result cannot be reproduced. Please report these details and, ideally, run each configuration multiple times with different seeds to provide variance estimates.
  5. [§4.3, Table 3] Several reported differences are within a small margin (e.g., Table 3, HM column: LV-FSC 36.59 vs. QW-FSC 35.13 for BERT; many cells differ by less than 1 F1 point). None of the comparisons include confidence intervals, significance tests, or repeated-seed variance. Claims such as "CoVExFiL significantly outperformed prior experiments" (§4.3) are therefore not statistically supported. Please add significance tests (e.g., paired bootstrap or approximate randomization) or at least report standard deviations over multiple runs.
minor comments (6)
  1. [Abstract and §1, §4.3] The reported improvement for SN is given as 26.24% in the abstract, 26.10% in the Introduction, and 26.14% in §4.3; the OF gain is 3.52% in the abstract and 3.58% in §4.3. Please harmonize these numbers.
  2. [§4.3 and Fig. 4] The text refers to "three-step CoT prompting," but the prompt template in Fig. 4 lists Step 1 through Step 4. Please clarify the intended number of steps and align the description.
  3. [§B.5] The paper states that "We also evaluated the performance of the models in the average precision score," but no average precision results appear anywhere in the paper. Either report these results or remove the claim.
  4. [§C.1] The correlation between BERTScore differences and classification performance is asserted qualitatively ("We observe a correlation") without a correlation coefficient or test. Please provide a quantitative measure.
  5. [Tables 1–3] The rows "Avg." and "Std." are not defined; state whether the standard deviation is across the four VLMs or across runs.
  6. [§B.3 and Appendix B.1] The paper does not specify the train/validation/test split sizes for Memotion or MAMI, nor the number of few-shot examples k beyond "k=1" implied by Fig. 4. Please make the experimental setup fully explicit.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CoVExFiL is a standard distillation pipeline evaluated on held-out labels; the test-set-based configuration selection and missing OCR-only control are evaluation concerns, not circular reductions.

full rationale

The paper's derivation chain is a three-part empirical study: (1) prompt four open-source VLMs on Memotion/MAMI; (2) LoRA fine-tune the VLMs; (3) generate VLM 'meme understanding' texts and fine-tune BERT/RoBERTa/XLNet on those texts with ground-truth labels. Evaluation is done with weighted F1 against held-out test labels from standard benchmarks, and the SOTA numbers are taken from external published models; no model weights are trained on the test labels, although the best (VLM, prompt, LLM) cell per task is selected after inspecting test-set results. The reference list contains no work by the present authors, so no self-citation chain is load-bearing, and no uniqueness theorem or ansatz is imported from prior author work. Two concerns are real but do not constitute circularity: the per-task best-configuration selection inflates reported gains, and the CoT prompt template (Fig. 4) ends with 'Conclude with the classification based on your reasoning,' so if the VLM's final label is not stripped from the 'meme understanding text' used in Exp 3, the LLM input may contain the VLM's own prediction; no ablation isolates the explanation's contribution over that embedded label. These are missing-control and evaluation-protocol issues, not a reduction of the claimed result to its inputs by construction.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim rests primarily on empirical comparisons. The main untested assumptions are the reliability of the ground truth and the sufficiency of VLM explanations for downstream classification. The external SOTA comparison is an ad hoc assumption that affects the reported improvement percentages.

free parameters (7)
  • LoRA rank = 16
    Chosen by hand for adapter fine-tuning of VLMs; affects capacity.
  • LoRA alpha = 32
    Scaling factor for LoRA updates, chosen by hand.
  • Learning rate = 2e-4
    Optimizer hyperparameter for LoRA fine-tuning.
  • Training epochs = 2
    Number of epochs for LoRA fine-tuning.
  • VLM generation temperature = 0.1
    Set low for reproducibility of generated explanations.
  • Few-shot example count k = 1
    One support example per class in FS/FSC prompts.
  • CoT prompt design = 4-step template (called three-step)
    The specific reasoning prompt that yielded best explanations; selected based on downstream performance.
assumptions (5)
  • domain assumption Memotion and MAMI annotations are reliable ground truth for the classification tasks.
    The evaluation treats crowd-sourced labels as correct; no adjudication or quality analysis is provided (Section B.1).
  • domain assumption VLM-generated explanations preserve the information needed for classification.
    CoVExFiL trains text-only classifiers on these explanations; any visual or contextual information omitted by the VLM is unrecoverable (Section 3.3 and error analysis C.3).
  • ad hoc to paper SOTA numbers from prior papers are comparable to results obtained here.
    Improvement percentages are computed against external SOTA values without re-running those models under identical splits (Tables 1-3, Section 4.3).
  • ad hoc to paper GPT-4 generated explanations are a valid silver standard for evaluating explanation quality.
    Used as references for BLEU/ROUGE/BERTScore comparison of VLM explanations (Section C.1).
  • domain assumption Weighted F1 is the appropriate metric for class-imbalanced meme classification.
    Chosen to counter class imbalance; other metrics (e.g., macro F1) might change conclusions (Section B.5).

how reviews work

0 comments
Cite this review

Pith. "Pith review of On VLMs for Diverse Tasks in Multimodal Meme Classification." pith.science (2026). https://pith.science/paper/JNV4KZGG

@misc{pith2026250520937,
  author       = {Pith},
  title        = {Pith review of: On VLMs for Diverse Tasks in Multimodal Meme Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JNV4KZGG}},
  note         = {Machine review of arXiv:2505.20937}
}
read the original abstract

In this paper, we present a comprehensive and systematic analysis of vision-language models (VLMs) for disparate meme classification tasks. We introduced a novel approach that generates a VLM-based understanding of meme images and fine-tunes the LLMs on textual understanding of the embedded meme text for improving the performance. Our contributions are threefold: (1) Benchmarking VLMs with diverse prompting strategies purposely to each sub-task; (2) Evaluating LoRA fine-tuning across all VLM components to assess performance gains; and (3) Proposing a novel approach where detailed meme interpretations generated by VLMs are used to train smaller language models (LLMs), significantly improving classification. The strategy of combining VLMs with LLMs improved the baseline performance by 8.34%, 3.52% and 26.24% for sarcasm, offensive and sentiment classification, respectively. Our results reveal the strengths and limitations of VLMs and present a novel strategy for meme understanding.

Figures

Figures reproduced from arXiv: 2505.20937 by the authors.

Figure 1
Figure 1. Schematic diagram of the three strategies mentioned in the experiment section. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Data samples from the Memotion dataset. For each meme, the full set of subcategories corresponding to [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Example from the MAMI dataset illustrating two tasks: Task A (binary: 1 = misogynistic, 0 = non [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Example Prompt Template for Experiment 1. Here, we have specified the prompts we used in ZS, ZSC, [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Examples from the test set with their corresponding gold labels are shown to illustrate the VLM’s [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: BERT Score Difference between correctly classified vs misclassified samples [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 26 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  4. [4]

    Tariq Habib Afridi, Aftab Alam, Muhammad Numan Khan, Jawad Khan, and Young-Koo Lee. 2021. A multimodal memes classification: A survey and open research issues. In Innovations in Smart Cities Applications Volume 4: The Proceedings of the 5th International Conference on Smart City Applications, pages 1451--1466. Springer

    work page 2021

  5. [5]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. 2024. https://arxiv.org/abs/2409.12191 Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution . arXiv preprint arXiv:2409.12191

    arXiv 2024

  6. [6]

    Prithvi Bhattacharya. 2019. Social degeneration through social media: A study of the adverse impact of ‘memes’. 2019 Sixth HCT Information Technology Trends (ITT), pages 44--46

    work page 2019

  7. [7]

    Efrat Blaier, Itzik Malkiel, and Lior Wolf. 2021. Caption enriched samples for improving hateful memes detection. arXiv preprint arXiv:2109.10649

    work page Pith review arXiv 2021

  8. [8]

    Mu Cai, Zeyi Huang, Yuheng Li, Utkarsh Ojha, Haohan Wang, and Yong Jae Lee. 2025. An investigation on llms' visual understanding ability using svg for image-text bridging. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 5377--5386. IEEE

    work page 2025

Show all 55 references
  1. [9]

    Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020. Uniter: Universal image-text representation learning. In European conference on computer vision, pages 104--120. Springer

    2020

  2. [10]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. https://lmsys.org/blog/2023-03-30-vicuna/ Vicuna: An open-source chatbot impressing gpt-4 with 90\

    2023

  3. [11]

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1--53

    2024

  4. [12]

    Shiv Shankar Das. 2023. Rising popularity of internet memes in india: A media psychology perspective. Indian Journal of Social Science and Literature, 2(3):5--9

    2023

  5. [13]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human...

    2019

  6. [14]

    Elisabetta Fersini, Francesca Gasparini, Giulia Rizzi, Aurora Saibene, Berta Chulvi, Paolo Rosso, Alyssa Lees, and Jeffrey Sorensen. 2022. Semeval-2022 task 5: Multimedia automatic misogyny identification. In Proceedings of the 16th International Workshop on Semantic Evaluatio...

    2022

  7. [15]

    Katharine Gelber and Luke McNamara. 2016. Evidencing the harms of hate speech. Social Identities, 22(3):324--341

    2016

  8. [16]

    Ming Shan Hee, Shivam Sharma, Rui Cao, Palash Nandi, Tanmoy Chakraborty, and Roy Ka-Wei Lee. 2024. Recent advances in hate speech moderation: Multimodality and the role of large models. arXiv preprint arXiv:2401.16727

    2024 arXiv

  9. [17]

    Lisa Anne Hendricks, Ronghang Hu, Trevor Darrell, and Zeynep Akata. 2018. Grounding visual explanations. In Proceedings of the European conference on computer vision (ECCV), pages 264--279

    2018

  10. [18]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3

    2022

  11. [19]

    Prince Jha, Raghav Jain, Konika Mandal, Aman Chadha, Sriparna Saha, and Pushpak Bhattacharyya. 2024. Memeguard: An llm and vlm-based framework for advancing content moderation via meme intervention. arXiv preprint arXiv:2406.05344

    2024 arXiv

  12. [20]

    Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. arXiv preprint arXiv:1707.07328

    2017 arXiv

  13. [21]

    Gitanjali Kumari, Dibyanayan Bandyopadhyay, Asif Ekbal, Santanu Pal, Arindam Chatterjee, and Vinutha BN. 2024. Let's all laugh together: A novel multitask framework for humor detection in internet memes. IEEE Transactions on Computational Social Systems, 11(3):4385--4395

    2024

  14. [22]

    Gwang Gook Lee and Mingwei Shen. 2022. Multi-modal, multi-task learning for memotion 2.0 challenge

    2022

  15. [23]

    Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. 2019. Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557

    2019 arXiv

  16. [24]

    Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, et al. 2020. Oscar: Object-semantics aligned pre-training for vision-language tasks. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August...

    2020

  17. [25]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74--81

    2004

  18. [26]

    Chen Liu, Gregor Geigle, Robin Krebs, and Iryna Gurevych. 2022. Figmemes: A dataset for figurative language identification in politically-opinionated memes. In Proceedings of the 2022 conference on empirical methods in natural language processing, pages 7069--7086

    2022

  19. [27]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024 a . Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296--26306

    2024

  20. [28]

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024 b . Llavanext: Improved reasoning, ocr, and world knowledge

    2024

  21. [29]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

    2019 arXiv

  22. [30]

    Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in neural information processing systems, 32

    2019

  23. [31]

    Khoi Nguyen and Vincent Ng. 2024. Computational meme understanding: A survey. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21251--21267

    2024

  24. [32]

    Hongliang Pan, Zheng Lin, Peng Fu, Yatao Qi, and Weiping Wang. 2020. Modeling intra and inter-modality incongruity for multi-modal sarcasm detection. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1383--1392

    2020

  25. [33]

    Yulia Petrova. 2021. Meme language, its impact on digital culture and collective thinking. In E3S Web of Conferences, volume 273, page 11026. EDP Sciences

    2021

  26. [34]

    Matt Post. 2018. A call for clarity in reporting bleu scores. arXiv preprint arXiv:1804.08771

    2018 arXiv

  27. [35]

    Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. 2021. Momenta: A multimodal framework for detecting harmful memes and their targets. arXiv preprint arXiv:2109.05184

    2021 arXiv

  28. [36]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  29. [37]

    Sathyanarayanan Ramamoorthy, Nethra Gunti, Shreyash Mishra, S Suryavardan, Aishwarya Reganti, Parth Patwa, Amitava DaS, Tanmoy Chakraborty, Amit Sheth, Asif Ekbal, et al. 2022. Memotion 2: Dataset on sentiment and emotion analysis of memes. In Proceedings of De-Factify: Worksh...

    2022

  30. [38]

    Siddhant Bikram Shah, Shuvam Shiwakoti, Maheep Chaudhary, and Haohan Wang. 2024. Memeclip: Leveraging clip representations for multimodal meme classification. arXiv preprint arXiv:2409.14703

    2024 arXiv

  31. [39]

    Chhavi Sharma, Deepesh Bhageria, William Scott, Srinivas Pykl, Amitava Das, Tanmoy Chakraborty, Viswanath Pulabaigari, and Bjorn Gamback. 2020. Semeval-2020 task 8: Memotion analysis--the visuo-lingual metaphor! arXiv preprint arXiv:2008.03781

    2020 arXiv

  32. [40]

    Shivam Sharma, Siddhant Agarwal, Tharun Suresh, Preslav Nakov, Md Shad Akhtar, and Tanmoy Chakraborty. 2023. What do you meme? generating explanations for visual semantic role labelling in memes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, page...

    2023

  33. [41]

    female astronaut: Because sandwiches won’t make themselves up there

    Smriti Singh, Amritha Haridasan, and Raymond Mooney. 2023. “female astronaut: Because sandwiches won’t make themselves up there”: Towards multimodal misogyny detection in memes. In The 7th Workshop on Online Abuse and Harms (WOAH), pages 150--159

    2023

  34. [42]

    Shardul Suryawanshi, Bharathi Raja Chakravarthi, Mihael Arcan, and Paul Buitelaar. 2020. Multimodal meme dataset (multioff) for identifying offensive content in image and text. In Proceedings of the second workshop on trolling, aggression and cyberbullying, pages 32--41

    2020

  35. [43]

    Chris Van der Lee, Albert Gatt, Emiel Van Miltenburg, and Emiel Krahmer. 2021. Human evaluation of automatically generated text: Current trends and best practice guidelines. Computer Speech & Language, 67:101151

    2021

  36. [44]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  37. [45]

    Jialu Xing, Jianping Liu, Jian Wang, Lulu Sun, Xi Chen, Xunxun Gu, and Yingfei Wang. 2024. A survey of efficient fine-tuning methods for vision-language models—prompt and adapter. Computers & Graphics, 119:103885

    2024

  38. [46]

    Bo Xu, Tingting Li, Junzhe Zheng, Mehdi Naseriparsa, Zhehuan Zhao, Hongfei Lin, and Feng Xia. 2022. Met-meme: A multimodal meme dataset rich in metaphors. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval, pages ...

    2022

  39. [47]

    Yanzhi Xu, Yueying Hua, Shichen Li, and Zhongqing Wang. 2024. https://doi.org/10.18653/v1/2024.acl-long.6 Exploring chain-of-thought for multi-modal metaphor detection . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long ...

    2024 doi

  40. [48]

    Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems, 32

    2019

  41. [49]

    Francisco Yus. 2019. Multimodality in memes: A cyberpragmatic approach. Analyzing digital discourse: New insights and future directions, pages 105--131

    2019

  42. [50]

    Jing Zhang and Yujin Wang. 2022. Srcb at semeval-2022 task 5: Pretraining based image to text late sequential fusion system for multimodal misogynous meme identification. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 585--596

    2022

  43. [51]

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675

    2019 arXiv

  44. [52]

    Yazhou Zhang, Jinglin Wang, Yaochen Liu, Lu Rong, Qian Zheng, Dawei Song, Prayag Tiwari, and Jing Qin. 2023. A multitask learning model for multimodal sarcasm, sentiment and emotion recognition in conversations. Information Fusion, 93:282--301

    2023

  45. [53]

    Yang Zhong and Bhiman Kumar Baghel. 2024. Multimodal understanding of memes with fair explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2007--2017

    2024

  46. [54]

    Qiji Zhou, Ruochen Zhou, Zike Hu, Panzhong Lu, Siyang Gao, and Yue Zhang. 2024. Image-of-thought prompting for visual reasoning refinement in multimodal large language models. arXiv preprint arXiv:2405.13872

    2024 arXiv

  47. [55]

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.