Pith. sign in

REVIEW 4 major objections 6 minor 53 references

Decoding Memes: Benchmarking Narrative Role Classification across Multilingual and Multimodal Models

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Across 25+ models, a benchmark finds no current model reliably identifies whether a meme casts an entity as hero, villain, victim, or other.

desk verdict Useful benchmark with a real negative result on Victim, but the abstract overstates prompt-engineering gains and the evaluation never quantifies OCR/label noise. read the letter →

arxiv 2506.23122 v1 pith:AQN3LMD2 submitted 2025-06-29 cs.CL cs.CY

Shivam Sharma , Tanmoy Chakraborty This is my paper · ORCID
classification cs.CLcs.CY
keywords narrativeroleclassificationmemeanalysismultimodalreasoningcode-mixedHinglishvictimdetectionbenchmarkpromptengineeringvision-languagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish a benchmark for a hard, under-studied task: deciding whether a person or group shown in a meme is cast as hero, villain, victim, or other, in English and code-mixed English-Hindi memes. It assembles 2,850 annotated examples across three test sets and evaluates more than 25 models, from fine-tuned multilingual transformers to text-only and vision-language LLMs in zero-shot settings. The central finding is that no tested model robustly performs the task: the victim role is consistently the most difficult, with F1 scores often below 0.35 even for the best systems, and performance drops on culturally specific and code-mixed content. The paper also shows that adding structured role definitions to prompts helps multimodal models only marginally. If this diagnosis is right, current automated tools cannot be trusted to infer who is framed as harmed in a meme without substantial task-specific adaptation.

What carries the argument

The load-bearing machinery is a four-class annotation scheme (Hero, Villain, Victim, Other) applied to three meme test sets, combined with a uniform evaluation protocol in which OCR-extracted text, a queried entity, and prompt instructions are fed to each model and scored with macro-averaged precision, recall, and F1. The paper's four prompt variants (basic, other-emphasized, definition-optimized, and hybrid) function as the controlled intervention for multimodal models, while confusion matrices and embedding visualizations expose which classes collapse into which. The central contrast the comparison relies on is between real meme text, which is culture-specific and indirect, and synthetic hateful meme text, which exhibits explicit and repetitive lexical markers.

What would settle it

A replication with a second round of annotations on a random sample of all three test sets, plus manual transcription of meme text instead of OCR, could settle the claim: if victim F1 jumps substantially under cleaner labels and transcripts, the reported difficulty is partly an artifact of annotation or OCR noise; if it stays near 0.2, the model failure is genuine.

Watch

Extended reading notes

Core claim

On the author's own terms, the discovery is that narrative-role classification in memes is not solved by scale or multimodality alone. On the original HVVMemes test set (US politics and COVID-19, skewed toward the other class), the fine-tuned DeBERTa-v3-large model reaches the best macro F1 of 0.54, while the strongest zero-shot instruction-tuned LLM, Qwen2.5-7B-Instruct, reaches 0.37 and the best multimodal model, LLaVA-NeXT with a definition-rich prompt, reaches 0.33. On the more balanced English HVVMemes-2 set, Qwen2.5-VL-7B-Instruct leads with macro F1 0.35; on the code-mixed English-Hindi set, Qwen2.5-VL and Qwen2-VL both reach 0.30. Across all settings the victim class is the recurring failure point, with F1 values exceeding 0.4 only rarely and falling below 0.1 in many configurations, while hero detection also trails the random baseline in several settings. The paper interprets this as evidence that subtle, culturally grounded framing requires better multimodal reasoning, cultural grounding, and role-adaptive training rather than simply larger models.

Load-bearing premise

The benchmark assumes the human-annotated role labels are correct ground truth and that OCR-extracted text faithfully represents what each meme says; annotation mistakes or OCR errors would change every F1 score and every model ranking without any model changing.

Editorial extensions

If this is right

  • The best fine-tuned multilingual transformer (DeBERTa-v3-large, macro F1 0.54 on HVVMemes) still beats the best zero-shot LLMs and multimodal models, so model scale alone does not close the gap.
  • Victim detection is the recurring bottleneck: across all three test sets and most model families, victim F1 rarely exceeds 0.4 and often falls below 0.1, so any deployed system should treat victim outputs as unreliable.
  • Code-mixed English-Hindi memes form a linguistically and culturally distinct cluster in embedding space, and most models trained on English or synthetic hate data do not transfer to them; the En-Hi test set is not just a translation of the English one.
  • For multimodal models, a hybrid prompt that combines role definitions with structured instructions yields small but consistent macro-F1 gains over the basic prompt, particularly for hero and villain roles.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the benchmark is accepted, a practical reading for deployment is that zero-shot role labelling is not production-ready: systems should be fine-tuned with role-adaptive objectives, and any victim-class output should be treated as low-confidence.
  • Because synthetic hateful memes carry an explicit and repetitive lexicon, models evaluated only on such data may look better than they are; real-meme performance is the harder bar and should be the primary evaluation surface.
  • A testable extension is to replace OCR with human-verified transcripts and to add inter-annotator agreement scores; if victim F1 jumps substantially under cleaner labels, part of the reported difficulty is data noise rather than model reasoning.
  • The success of Qwen2.5-VL on Hinglish examples appears to draw on cultural knowledge about public figures such as MS Dhoni; a benchmark that varies whether that background knowledge is supplied would directly test whether the bottleneck is cultural grounding or multimodal fusion.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper benchmarks narrative-role classification (Hero, Villain, Victim, Other) in memes across three test sets: the original HVVMemes (English) and the HVVMemes-2 test sets in English and code-mixed English-Hindi. It reports lexical and structural dataset analyses, evaluates over 25 text-only and multimodal models in zero-shot and fine-tuned settings, and studies four prompt variants for LLaVA-NeXT. The main empirical claims are that the Victim class is consistently hard to identify, that code-mixed and culturally specific content hurts generalisation, and that hybrid prompts offer marginal but consistent improvements.

Significance. If the central negative result holds, the paper is a useful benchmark contribution: it assembles a broad model suite, connects two existing shared-task datasets, and provides qualitative, embedding, and confusion-matrix analyses that are largely absent from the original dataset papers. The finding that even the best zero-shot multimodal models plateau at macro F1 around 0.30–0.35, with victim F1 often below 0.20, would be a valuable calibration point for the community. However, the headline conclusions depend on two unverified data-quality assumptions: the gold labels are treated as noise-free, and the OCR text is treated as a faithful representation of meme content. Because neither assumption is audited, the central claim is not yet established.

major comments (4)
  1. [III, V, VI.A] The central negative claim about Victim detection and code-mixed generalisation is not separable from annotation and OCR noise. Section III introduces HVVMemes and HVVMemes-2 without reporting inter-annotator agreement or label-quality statistics, and every prompt in Section V inserts {ocr} directly into the model input. The paper itself concedes an 'OCR-based performance bottleneck' in Section VI.A but never quantifies OCR accuracy or its downstream effect. Since the confusion-matrix analysis in Section VII.C shows Victim being absorbed into Other, a plausible alternative explanation is that the Victim label is the least stable under re-annotation and that OCR errors disproportionately affect code-mixed romanised text. The authors should provide annotation agreement, an OCR error analysis, or a re-evaluation on a manually verified subset before the global 'Victim remains hard' conclusion can be accepted.
  2. [Table I vs Tables II and III] There is a direct numerical inconsistency in the core result tables. Table I reports 1,084 total entity references in HVVMemes (49 Hero, 273 Villain, 95 Victim, 667 Other), but Tables II and III both show column totals of 2,433 with 1,917 instances of Other, and the majority-baseline precision of 0.79 in Table III is computed against 1,917/2,433. These numbers cannot describe the same test set. Since all macro-F1 comparisons, majority baselines, and class-wise conclusions in Table III rest on these counts, the authors must correct the headers or explain which superset the evaluations actually used. The same issue affects Table II, whose column headers are identical across rows for three different test sets.
  3. [Abstract, Contributions, V, Table II] The abstract and the Contributions section claim that hybrid prompts (P4) provide 'marginal yet consistent improvements', but the reported results contradict this. In Table II, on HVVMemes (En), P4 has macro F1 0.240, clearly below P3's 0.333; the same reversal appears in Table III. On HVVMemes-2 (EnHi), P4 is better than P3 overall, but it is still worse than P1 for Hero (0.310 vs 0.372) and for Villain (0.393 vs 0.468). The contribution statement that P4 'best generalises across roles, especially villain and hero' is therefore not supported by the data shown. The claim should be restricted to the settings where it holds, or the prompt analysis should be revised.
  4. [IV and VI] The headline comparison between fine-tuned transformers and zero-shot LLMs confounds architecture with training condition. In Table III, deberta-v3-large (fine-tuned on the task) reaches macro F1 0.543, while Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and the multimodal models are evaluated zero-shot. The paper later attributes DeBERTa's advantage to 'reasoning capabilities of current LLMs' being limited, but the more parsimonious reading is that fine-tuning on the training split explains much of the gap. The authors should either add a fine-tuned LLM comparison or clearly frame the result as 'best fine-tuned model vs best zero-shot model' and soften the architectural conclusions.
minor comments (6)
  1. [Throughout] The model name is written inconsistently as LLaVA-NeXT, LLaVA-Next, and LlaVA-Next; please standardise.
  2. [I] In the Introduction, 'Another ridiculous Rahul Gandhi's speech' should read 'Another meme ridicules Rahul Gandhi's speech'.
  3. [VI.C] In Section VI.C, 'performes' should be 'performs'.
  4. [Table II] Table II's column headers list counts that are identical for all three test-set rows; the table should instead show per-dataset class counts, e.g., 49/273/95/667 for HVVMemes (En).
  5. [VI.B] In Section VI.B, the text refers to 'Qwen-2.5B-Instruct', but the model evaluated is listed as Qwen2.5-7B-Instruct; please correct the name.
  6. [V] The paper does not report decoding parameters (temperature, top-p, number of runs, seed) for the instruction-tuned and multimodal LLMs; without this, differences such as 0.295 vs 0.242 in Table II may reflect sampling noise rather than prompt quality.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; the paper is an empirical benchmark against public datasets with independent majority/random baselines.

full rationale

The paper's central claims are empirical generalizations from F1 scores of over 25 models evaluated on HVVMemes and HVVMemes-2. No model parameter is fitted to the test labels and then reported as a prediction; the role taxonomy, prompts, and evaluation protocol come from prior work and shared tasks, not from the results themselves. The datasets are cited to [5] and [6] rather than re-derived, and although some of those citations include the present authors, they are public shared-task benchmarks used as external ground truth. The paper also reports majority and random baselines that independently anchor the difficulty of the task. The remaining concerns—unmeasured OCR noise and the absence of inter-annotator agreement—are validity threats that would affect any evaluation on these data, but they do not make any claim true by construction. The frequent self-citations document dataset provenance and related work rather than supplying an unverified load-bearing theorem, so no circularity is present.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central analysis rests on dataset labels, OCR text, and the four-class annotation scheme; the paper introduces no numeric free parameters or invented entities. All assumptions are domain assumptions about measurement validity, not mathematical axioms.

assumptions (4)
  • domain assumption Human annotations in HVVMemes and HVVMemes-2 correctly identify the narrative role of each entity.
    Section III adopts the labels from [5] and [6] without reporting inter-annotator agreement or adjudication stats; all F1 scores are computed against these labels.
  • domain assumption OCR-extracted text faithfully captures the embedded text of each meme.
    Prompt templates insert {ocr} as embedded text (Appendix A), and lexical analyses are based on this text; OCR errors would propagate into all results.
  • domain assumption The four narrative roles (hero, villain, victim, other) are mutually exclusive and exhaustive as annotated.
    The classification setup in Section V requires exactly one label per entity; ambiguity between victim and other is later identified as a challenge, suggesting the premise may be strained.
  • domain assumption Zero-shot prompting provides a valid measurement of model capability for this task.
    The paper does not calibrate prompts for each model or verify that each model's output distribution over labels is meaningful; it assumes chat-style instruction following is a fair probe (Section IV).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Decoding Memes: Benchmarking Narrative Role Classification across Multilingual and Multimodal Models." pith.science (2026). https://pith.science/paper/AQN3LMD2

@misc{pith2026250623122,
  author       = {Pith},
  title        = {Pith review of: Decoding Memes: Benchmarking Narrative Role Classification across Multilingual and Multimodal Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AQN3LMD2}},
  note         = {Machine review of arXiv:2506.23122}
}
read the original abstract

This work investigates the challenging task of identifying narrative roles - Hero, Villain, Victim, and Other - in Internet memes, across three diverse test sets spanning English and code-mixed (English-Hindi) languages. Building on an annotated dataset originally skewed toward the 'Other' class, we explore a more balanced and linguistically diverse extension, originally introduced as part of the CLEF 2024 shared task. Comprehensive lexical and structural analyses highlight the nuanced, culture-specific, and context-rich language used in real memes, in contrast to synthetically curated hateful content, which exhibits explicit and repetitive lexical markers. To benchmark the role detection task, we evaluate a wide spectrum of models, including fine-tuned multilingual transformers, sentiment and abuse-aware classifiers, instruction-tuned LLMs, and multimodal vision-language models. Performance is assessed under zero-shot settings using precision, recall, and F1 metrics. While larger models like DeBERTa-v3 and Qwen2.5-VL demonstrate notable gains, results reveal consistent challenges in reliably identifying the 'Victim' class and generalising across cultural and code-mixed content. We also explore prompt design strategies to guide multimodal models and find that hybrid prompts incorporating structured instructions and role definitions offer marginal yet consistent improvements. Our findings underscore the importance of cultural grounding, prompt engineering, and multimodal reasoning in modelling subtle narrative framings in visual-textual content.

Figures

Figures reproduced from arXiv: 2506.23122 by the authors.

Figure 1
Figure 1. Examples of memes used, representing (a) [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Embedded text length distribution for HVVMemes (En) – left, HVVMemes-2 (En) – middle and HVVMemes-2 (EnHi) – right. TABLE I Role-wise counts of the entity-references as different roles in HVVMemes [5] and HVVMemes-2 [6] datasets. Roles EN EN EN-HI Hero 49 144 252 Villain 273 404 348 Victim 95 122 207 Other 667 141 148 Total 1084 811 955 task. III. Dataset The dataset introduced in [5] incorporated narrative role lab… view at source ↗
Figure 3
Figure 3. Qualitative examples for HVV-P1-EN, top 3 models from each category. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: t-SNE plot visualising the data points from the three [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Confusion matrices for the top-performing multimodal models across different test sets. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Word cloud representations for HVVMemes (En) – left, HVVMemes-2 (En) – middle, and HVVMemes-2 (EnHi) – right. the suite of baseline models used, spanning multilingual, code-mixed, sentiment, vision-language, and instruction￾tuned architectures. Comprehensive error anal…
Figure 7
Figure 7. Figure 7: Confusion matrices across top three performant models across different baseline categories and test sets. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Qualitative examples for HVV-P2-EN, top three models from each category. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Qualitative examples for HVV-P2-CM, top three models from each category. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 45 canonical work pages

  1. [1]

    Hate speech detection: Challenges and solutions,

    S.MacAvaney,H.-R.Yao,E.Yang,K.Russell,N.Goharian,and O. Frieder, “Hate speech detection: Challenges and solutions,” PLOS ONE, vol. 14, no. 8, pp. 1–16, 08 2019. [Online]. Available: https://doi.org/10.1371/journal.pone.0221152

    work page doi:10.1371/journal.pone.0221152 2019

  2. [2]

    The hateful memes challenge: Detecting hate speech in multimodal memes,

    D. Kiela, H. Firooz, A. Mohan et al., “The hateful memes challenge: Detecting hate speech in multimodal memes,” in Proceedings of the 34th International Conference on Neural InformationProcessingSystems , ser. NeurIPS ’20, vol. 33, 2020

    work page 2020

  3. [3]

    Batman, Pandaman and the Blind Man: A case study in social change memes and internet censorship in China,

    A. X. Mina, “Batman, Pandaman and the Blind Man: A case study in social change memes and internet censorship in China,” Journal of Visual Culture, vol. 13, no. 3, pp. 359–375, 2014. [Online]. Available: https://doi.org/10.1177/1470412914546576

    work page doi:10.1177/1470412914546576 2014

  4. [4]

    Detecting harmful memes and their targets,

    S. Pramanick, D. Dimitrov, R. Mukherjee et al., “Detecting harmful memes and their targets,” in Findings of ACL, ser. ACL-IJCNLP ’21, aug 2021, pp. 2783–2796

    work page 2021

  5. [5]

    Findings of the constraint 2022 shared task on detecting the hero, the villain, and the victim in memes,

    S. Sharma, T. Suresh, A. Kulkarni et al., “Findings of the constraint 2022 shared task on detecting the hero, the villain, and the victim in memes,” inProceedings of the Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situations, 2022, pp. 1–11

    work page 2022

  6. [6]

    The clef-2024 checkthat! lab: Check-worthiness, subjectivity, persuasion, roles, authori- ties, and adversarial robustness,

    A. Barrón-Cedeño, F. Alam, T. Chakraborty, T. Elsayed, P. Nakov, P. Przybyła, J. M. Struß, F. Haouari, M. Hasanain, F. Ruggeri, X. Song, and R. Suwaileh, “The clef-2024 checkthat! lab: Check-worthiness, subjectivity, persuasion, roles, authori- ties, and adversarial robustness,” inAdvances in Information 14 Retrieval, N. Goharian, N. Tonellotto, Y. He, A....

    work page 2024

  7. [7]

    SemEval-2020 task 8: Memotion analysis- the visuo-lingual metaphor!

    C. Sharma et al., “SemEval-2020 task 8: Memotion analysis- the visuo-lingual metaphor!” inProceedings of the Fourteenth Workshop on Semantic Evaluation, ser. SemEval ’20, 2020, pp. 759–773. [Online]. Available: https://www.aclweb.org/ anthology/2020.semeval-1.99

    work page 2020

  8. [8]

    Sarc-M: Sarcasm detection in typo- graphic memes,

    A. Kumar and G. Garg, “Sarc-M: Sarcasm detection in typo- graphic memes,” inProceedings of the International Conference on Advances in Engineering Science Management & Technol- ogy, ser. ICAESMT ’19, Dehradun, India, 2019

    work page 2019

Show all 53 references
  1. [9]

    Multimodal learning for hateful memes detection,

    Y. Zhou, Z. Chen, and H. Yang, “Multimodal learning for hateful memes detection,” inProceedings of the International Conference on Multimedia Expo Workshops, ser. ICMEW ’21, 2021, pp. 1–6

    2021

  2. [10]

    Emotion- aware multimodal fusion for meme emotion detection,

    S. Sharma, R. S, M. S. Akhtar, and T. Chakraborty, “Emotion- aware multimodal fusion for meme emotion detection,”IEEE Transactions on Affective Computing, pp. 1800–1811, 2024

    2024

  3. [11]

    DISARM: Detecting the victims targeted by harmful memes,

    S. Sharma, M. S. Akhtar, P. Nakov, and T. Chakraborty, “DISARM: Detecting the victims targeted by harmful memes,” in Findings of NAACL, Seattle, Washington, USA, 2022

    2022

  4. [12]

    Characterizing the entities in harmful memes: Who is the hero, the villain, the victim?

    S. Sharma, A. Kulkarni, T. Sureshet al., “Characterizing the entities in harmful memes: Who is the hero, the villain, the victim?” inProceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics , A. Vlachos and I. Augenstein, Eds....

    2023

  5. [13]

    SemEval-2025 task 10: Multilingual char- acterization and extraction of narratives from online news,

    J. Piskorski et al., “SemEval-2025 task 10: Multilingual char- acterization and extraction of narratives from online news,” in Proceedings of the 19th International Workshop on Semantic Evaluation, ser. SemEval 2025, Vienna, Austria, July 2025

    2025

  6. [14]

    Logically at the CON- STRAINT 2022: Multimodal role labelling,

    L. Kun, J. Bankoti, and D. Kiskovski, “Logically at the CON- STRAINT 2022: Multimodal role labelling,” in Proceedings of the Workshop on Combating Online Hostile Posts in Re- gional Languages during Emergency Situations , ser. CON- STRAINT ’22, Dublin, Ireland, 2022

    2022

  7. [15]

    DD-TIGatCon- straint@ACL2022: Multimodal understanding and reasoning for role labeling of entities in hateful memes,

    Z.Zhou,H.Zhao,J.Dong,J.Gao,andX.Liu,“DD-TIGatCon- straint@ACL2022: Multimodal understanding and reasoning for role labeling of entities in hateful memes,” inProceedings of the Workshop on Combating Online Hostile Posts in Regional Lan- guages during Emergency Situations, ser. ...

    2022

  8. [16]

    Detecting and understanding harmful memes: A survey,

    S. Sharma, F. Alam, M. S. Akhtar et al., “Detecting and understanding harmful memes: A survey,” inProceedings of the 31st International Joint Conference on Artificial Intelligence, ser. IJCAI-ECAI ’22, Vienna, Austria, 2022

    2022

  9. [17]

    Recent advances in online hate speech moderation: Multimodality and the role of large models,

    M. S. Hee, S. Sharma, R. Cao, P. Nandi, P. Nakov, T. Chakraborty, and R. K.-W. Lee, “Recent advances in online hate speech moderation: Multimodality and the role of large models,” in Findings of the Association for Computational Linguistics: EMNLP 2024, Y. Al-Onaizan, M. Bansa...

    2024

  10. [18]

    Race, ethnicity and national origin-based discrimination in social media and hate crimes across 100 u.s. cities,

    K. Relia, Z. Li, S. H. Cook, and R. Chunara, “Race, ethnicity and national origin-based discrimination in social media and hate crimes across 100 u.s. cities,”ICWSM, vol. 13, no. 01, pp. 417–427, Jul. 2019

    2019

  11. [19]

    Anyone can become a troll: Causes of trolling behavior in online discussions,

    J. Cheng, M. Bernstein, C. Danescu-Niculescu-Mizil, and J. Leskovec, “Anyone can become a troll: Causes of trolling behavior in online discussions,” in Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, ser. CSCW ’17, New York, ...

    2017

  12. [20]

    A survey on computational propa- ganda detection,

    G. Da San Martino, S. Cresci, A. Barrón-Cedeño, S. Yu, R. Di Pietro, and P. Nakov, “A survey on computational propa- ganda detection,” inIJCAI, Yokohama, Japan, 2020, pp. 4826– 4832

    2020

  13. [21]

    #MeTooMA: Multi-aspect annotations of tweets related to the MeToo movement,

    A. Gautam, P. Mathur, R. Gosangi, D. Mahata, R. Sawhney, and R. R. Shah, “#MeTooMA: Multi-aspect annotations of tweets related to the MeToo movement,” Proceedings of the International AAAI Conference on Web and Social Media , vol. 14, no. 1, pp. 209–216, May 2020. [Online]. Av...

    2020

  14. [22]

    Multilingual and multi-aspect hate speech analysis,

    N. Ousidhoum, Z. Lin, H. Zhang, Y. Song, and D.-Y. Yeung, “Multilingual and multi-aspect hate speech analysis,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (...

    2019

  15. [23]

    Evaluating aspect- based sentiment classification on Twitter hate speech using neural networks and word embedding features,

    N. Zainuddin, A. Selamat, and R. Ibrahim, “Evaluating aspect- based sentiment classification on Twitter hate speech using neural networks and word embedding features,” inNew Trends in Intelligent Software Methodologies, Tools and Techniques, 2018, pp. 723–734

    2018

  16. [24]

    Targets and aspects in social media hate speech,

    A. Shvets, P. Fortuna, J. Soler, and L. Wanner, “Targets and aspects in social media hate speech,” in Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021), Online, Aug. 2021, pp. 179–190. [Online]. Available: https://aclanthology.org/2021.woah-1.19

    2021

  17. [25]

    Hatexplain: A benchmark dataset for explainable hate speech detection,

    B. Mathew et al., “Hatexplain: A benchmark dataset for explainable hate speech detection,” Proceedings of the AAAI Conference on Artificial Intelligence , vol. 35, no. 17, pp. 14867–14875, May 2021. [Online]. Available: https://ojs.aaai. org/index.php/AAAI/article/view/17745

    2021

  18. [26]

    Social bias frames: Reasoning about social and power implications of language,

    M. Sap, S. Gabriel, L. Qin, D. Jurafsky, N. A. Smith, and Y. Choi, “Social bias frames: Reasoning about social and power implications of language,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ser. ACL ’20, Online, Jul. 2020, pp. ...

    2020

  19. [27]

    Joint learning for targeted sentiment analysis,

    D. Ma, S. Li, and H. Wang, “Joint learning for targeted sentiment analysis,” inProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, Oct.-Nov. 2018, pp. 4737–4742. [Online]. Available: https://aclanthology.org/D18-1504

    2018

  20. [28]

    Open domain targeted sentiment,

    M. Mitchell, J. Aguilar, T. Wilson, and B. Van Durme, “Open domain targeted sentiment,” in Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Washington, USA, Oct. 2013, pp. 1643–

    2013

  21. [29]

    On the origins of memes by means of fringe web communities,

    S. Zannettou, T. Caulfield, J. Blackburnet al., “On the origins of memes by means of fringe web communities,” inProceedings of the Internet Measurement Conference 2018, ser. IMC ’18, New York, NY, USA, 2018, p. 188–202. [Online]. Available: https://doi.org/10.1145/3278532.3278550

    2018

  22. [30]

    A curious case of meme detection: An investigative study,

    C. Sharma. and V. Pulabaigari., “A curious case of meme detection: An investigative study,” inWEBIST, 2020, pp. 327– 338

    2020

  23. [31]

    Meme vs. non-meme classification using visuo-linguistic associa- tion,

    C. Sharma, V. Pulabaigari, and A. Das, “Meme vs. non-meme classification using visuo-linguistic associa- tion,” in International Conference on Web Information Systems and Technologies , 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:227129761

    2020

  24. [32]

    Multimodalmemedataset(MultiOFF)foridentifyingoffensive content in image and text,

    S.Suryawanshi,B.R.Chakravarthi,M.Arcan,andP.Buitelaar, “Multimodalmemedataset(MultiOFF)foridentifyingoffensive content in image and text,” inProceedings of the Second Work- shop on Trolling, Aggression and Cyberbullying, May 2020, pp. 32–41

    2020

  25. [33]

    Exploring hate speech detection in multimodal publications,

    R. Gomez, J. Gibert, L. Gomez, and D. Karatzas, “Exploring hate speech detection in multimodal publications,” inProceed- ings of the 2020 IEEE Winter Conference on Applications of Computer Vision, ser. WACV ’20, 2020, pp. 1459–1467

    2020

  26. [34]

    MOMENTA: A multimodal framework for detecting harmful memes and their targets,

    S. Pramanick, S. Sharma, D. Dimitrovet al., “MOMENTA: A multimodal framework for detecting harmful memes and their targets,” in Findings of EMNLP 2021, Nov. 2021, pp. 4439– 4455

    2021

  27. [35]

    KnowMeme: A knowledge-enriched graph neural network so- lution to offensive meme detection,

    L. Shang, C. Youn, Y. Zha, Y. Zhang, and D. Wang, “KnowMeme: A knowledge-enriched graph neural network so- lution to offensive meme detection,” in Proceedings of the 2021 IEEE 17th International Conference on eScience, ser. eScience ’21, 2021, pp. 186–195

    2021

  28. [36]

    Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation,

    K. Karkkainen and J. Joo, “Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation,” inWACV, 2021, pp. 1548–1558

    2021

  29. [37]

    MEMEX: Detecting explanatory evidence for memes via knowledge-enriched contextualization,

    S. Sharma, R. S, U. Arora, M. S. Akhtar, and T. Chakraborty, “MEMEX: Detecting explanatory evidence for memes via knowledge-enriched contextualization,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul. 202...

    2023

  30. [38]

    MemeMQA: Multimodal question answering for memes via rationale-based inferencing,

    S. Agarwal, S. Sharma, P. Nakov, and T. Chakraborty, “MemeMQA: Multimodal question answering for memes via rationale-based inferencing,” inFindings of the Association for Computational Linguistics: ACL 2024, L.-W. Ku, A. Martins, and V. Srikumar, Eds. Bangkok, Thailand: Associ...

    2024

  31. [39]

    GLUECoS: An evaluation benchmark for code-switched NLP,

    S. Khanuja, S. Dandapat, A. Srinivasanet al., “GLUECoS: An evaluation benchmark for code-switched NLP,” in ACL ’20, Online, Jul. 2020, pp. 3575–3585

    2020

  32. [40]

    Muril: Multilingual representations for indian languages,

    S. Khanuja, D. Bansal, S. Mehtaniet al., “Muril: Multilingual representations for indian languages,” 2021

    2021

  33. [41]

    BERT:Pre-trainingofdeepbidirectionaltrans- formers for language understanding,

    J.Devlin etal.,“BERT:Pre-trainingofdeepbidirectionaltrans- formers for language understanding,” inNAACL ’19’, 2019, pp. 4171–4186

    2019

  34. [42]

    BERT: pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: pre-training of deep bidirectional transformers for language understanding,” CoRR, vol. abs/1810.04805, 2018. [Online]. Available: http://arxiv.org/abs/1810.04805

    2018 arXiv

  35. [43]

    Data bootstrapping approaches to improve low-resource abusive language detection for indic languages,

    M. Das, S. Banerjee, and A. Mukherjee, “Data bootstrapping approaches to improve low-resource abusive language detection for indic languages,”arXiv preprint arXiv:2204.12543, 2022

    2022 arXiv

  36. [44]

    The llama 3 herd of models,

    A. Grattafiori et al., “The llama 3 herd of models,” 2024. [Online]. Available: https://arxiv.org/abs/2407.21783

    2024 arXiv

  37. [45]

    Mistral 7b,

    A. Q. Jiang, A. Sablayrolles, A. Menschet al., “Mistral 7b,”

  38. [46]

    Qwen2.5: A party of foundation models,

    Q. Team, “Qwen2.5: A party of foundation models,” September 2024. [Online]. Available: https://qwenlm.github.io/ blog/qwen2.5/

    2024

  39. [47]

    Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled em- bedding sharing,

    P. He, J. Gao, and W. Chen, “Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled em- bedding sharing,” 2021

    2021

  40. [48]

    Qwen2-vl: Enhancing vision- language model’s perception of the world at any resolution,

    P. Wang, S. Bai, S. Tanet al., “Qwen2-vl: Enhancing vision- language model’s perception of the world at any resolution,” arXiv preprint arXiv:2409.12191, 2024

    2024 arXiv

  41. [49]

    Qwen2.5-vl,

    Q. Team, “Qwen2.5-vl,” January 2025. [Online]. Available: https://qwenlm.github.io/blog/qwen2.5-vl/

    2025

  42. [50]

    Llava-next: Improved reasoning, ocr, and world knowledge,

    H. Liu et al., “Llava-next: Improved reasoning, ocr, and world knowledge,” January 2024. [Online]. Available: https: //llava-vl.github.io/blog/2024-01-30-llava-next/

    2024

  43. [51]

    Learning transferable visual models from natural language supervision,

    A. Radford et al., “Learning transferable visual models from natural language supervision,” inICML, 2021, pp. 8748–8763

    2021

  44. [1654]

    Available: https://aclanthology.org/D13-1171

    [Online]. Available: https://aclanthology.org/D13-1171

  45. [2023]

    Available: https://arxiv.org/abs/2310.06825

    [Online]. Available: https://arxiv.org/abs/2310.06825

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.