Pith. sign in

REVIEW 4 major objections 4 minor 74 references

The paper claims that hateful memes are defined by two jointly necessary features—presupposed context and false claims—and that SHIELD, a detector built on those features, outperforms prior models and transfers to fake-news detection.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 10:22 UTC pith:KEY4FGYI

load-bearing objection Solid architecture paper with real benchmark gains; just don't buy the psychological story as tested. the 4 major comments →

arxiv 2510.09935 v2 pith:KEY4FGYI submitted 2025-10-11 cs.CL cs.AI

Unpacking Hateful Memes: Presupposed Context and False Claims

Weibin Cai , Jiayu Li , Reza Zafarani This is my paper
classification cs.CL cs.AI
keywords hateful meme detectionpresupposed contextfalse claimscross-modal reference graphlarge language modelsmultimodal classificationfake news detectionsocial perception
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that a meme is hateful not because of any single slur or insult but because it combines two expressive features: a presupposed context that quietly casts one group as good and another as bad, and a false claim that misrepresents either the image or the target group. To test this account, the authors build SHIELD, with PCM modeling the presupposed context and FACT detecting false claims through socially informed perception and an explicit text-to-image reference graph. On three hateful-meme benchmarks the full system beats earlier models on AUC, accuracy, and macro-F1, and the same framework carries over to multimodal fake-news classification. The central wager is that these two features are the essence of hate expression in memes, so that modeling them directly is better than relying on generic pretrained representations.

Core claim

SHIELD rests on the claim that hateful memes are defined by two jointly necessary features. A presupposed evaluative context—the framing that marks an in-group as good and an out-group as bad—makes the meme's value judgment legible; a false claim, either an incorrect reference between text and image or a deliberately misleading stereotype, supplies the distortion that turns context into attack. The paper constructs PCM to fuse text and image context with a Hadamard product that preserves both consistent and ironic-inconsistent expressions, and FACT with SPM, which draws on a fine-tuned LLM's latent social knowledge, together with CRM, which builds a graph connecting text tokens to image patc

What carries the argument

PCM (Presupposed Context Module) encodes image patches and text tokens separately, then fuses them via a Hadamard product of linear projections so that aligned sentiment is amplified and ironic contrast becomes negative. FACT (False Claims Module) has two parts: SPM (Social Perception Module), which feeds the meme to a LoRA-fine-tuned LLM with the prompt 'Are there false claims?' and takes the last hidden state, and CRM (Cross-modal Reference Module), which uses the LLM's attention matrix to connect each text token to its top-K attended image patches, builds a graph with token-token, patch-patch, and token-patch edges, and pools a GCN's node embeddings into a graph representation. The concat

Load-bearing premise

The load-bearing assumption is that the language model's attention scores, learned only from the final hateful/non-hateful label, faithfully indicate which words in the meme refer to which parts of the image.

What would settle it

Build a small evaluation set with human-annotated text-token-to-image-patch links. If SHIELD's top-K attention edges mostly disagree with those human links while the model's accuracy stays high when the graph module is removed, then the graph is not detecting false claims through reference and the paper's explanation of its gains fails. A second check: if memes that clearly contain both a presupposed context and a false claim are consistently rated non-hateful by human readers, the two-feature account would need revision.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If SHIELD is right, hateful-meme detectors should be built to recognize presupposition and falsehood rather than to memorize surface patterns such as slurs or topics.
  • The explicit cross-modal reference graph gives a mechanism for 'incorrectness': a word that points to the wrong image region is itself evidence of a false claim.
  • The two-feature account predicts that neither presupposition alone nor falsehood alone should be enough for hate, which narrows the search for training signals and could reduce false positives on dark humor and satire.
  • Transfer to fake news suggests the same features may structure other forms of harmful multimodal content, so a single architecture may serve several content-moderation tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper's theory implies a testable ordering—memes containing both features should be judged more hateful than memes containing only one; a crowd-sourced rating study could verify this directly.
  • Editorial inference: because the reference graph is learned from binary labels rather than from annotated reference relations, the graph may encode task-specific shortcuts rather than genuine word-to-image reference; this could be probed by comparing attention edges with human-annotated reference links.
  • Editorial inference: the same modules could be applied to other multimodal misinformation formats, such as memes with audio or video, where presupposed context and false claims may appear in different modality pairings.
  • Editorial inference: the Hadamard fusion predicts a direction-sensitive context signal—consistent memes amplify, inconsistent memes go negative—so an intervention that decorrelates text and image sentiment should produce a measurable drop in PCM's contribution.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper argues from philosophy and psychology that hateful memes are characterized by presupposed context and false claims, and introduces SHIELD, which combines a Presupposed Context Module (PCM) and a False Claims Module (FACT) containing Social Perception and Cross-modal Reference Modules. It reports state-of-the-art results on FHM, Harm-C, and Harm-P across AUC, Accuracy, and Macro-F1 (Tables 2-3), generalization across hate targets (Table 4), and transfer to fake news classification (Table 5).

Significance. If the construct-validity concerns were resolved, this would be a valuable contribution: it connects a theoretical account of hate expression to a concrete architecture, evaluates on three benchmark datasets with ablations and parameter sensitivity, and demonstrates transfer to fake news. The paper's main strengths are the breadth of empirical evaluation and the explicit attempt to make the theoretical constructs operational. However, the current evidence does not establish that PCM and FACT actually measure presupposed context and false claims rather than serving as additional learned features.

major comments (4)
  1. [Section 4.2.2, Eqs. (5)-(6)] The cross-modal reference graph is constructed by taking top-K image-patch attention scores from an LLM that is fine-tuned only on binary hateful/non-hateful labels (Section 4.2.1). There is no reference annotation, no false-claim supervision, and no evidence that attention scores encode genuine token-patch referential links. Since the CRM is named a 'false claim' module and the paper's central claim is that SHIELD models false claims, this is a load-bearing construct-validity gap. The authors should validate the attention proxy directly (e.g., against human-annotated reference links or a diagnostic probe) or temper the interpretational claims.
  2. [Section 4.3 and Table 3] The ablation only removes whole modules; it does not test whether the graph structure or the 'false claim' prompt is responsible for the gains. The CRM-only contribution is small (e.g., +0.25 AUC and +0.71 Accuracy on FHM; +0.07 AUC on Harm-C; +0.84 AUC on Harm-P when comparing SPM+PCM with full SHIELD), which could be due to added GNN parameters or arbitrary graph structure rather than referential fidelity. A comparison with a random/parameter-matched graph or a non-graph MLP over the same node features is needed to support the claim that the CRM's explicit reference graph is what helps.
  3. [Section 4.4, Theorem 4.1 and Eq. (13)] The proof does not establish the sign-flip condition. The authors assume l > 0 and state that an ideal model should satisfy sign(Δl) ≠ sign(l) and |Δl| > |l|, but the Cauchy-Schwarz step only provides an upper bound on |Δl|. The condition in Eq. (10) guarantees |Δl| > |l|, not that Δl has the opposite sign of l. Indeed, Δl could have the same sign as l, in which case the prediction is reinforced rather than flipped. The theorem should be restated as a sensitivity/magnitude bound, or the sign condition must be derived (generally impossible with only the norm bound). This undermines the theoretical justification for the reference graph.
  4. [Sections 4.1-4.2 and Eq. (8)] Because PCM and SPM are trained end-to-end on the binary hateful/non-hateful label, their embeddings are not independently shown to correspond to presupposed context or false claims. The prompt 'Are there false claims?' is not a supervision signal; h_SP is just the last hidden state of a binary fine-tuned LLM. The paper therefore overstates the interpretive claim that SHIELD 'captures presupposed context and false claims.' At minimum, the claims should be softened to 'modules designed to model these features,' and ideally supported by human annotation or intervention experiments.
minor comments (4)
  1. [Appendix B, Table 6] The reported main result uses K=4, but Table 6 shows K=8 gives a higher FHM AUC (87.69 vs. 87.51). The model-selection rule for K should be stated explicitly; if K was chosen by validation performance, that choice should be documented.
  2. [Section 4.1] Typo: 'Hardamard product' should be 'Hadamard product.' Also, the two 'context types' (consistent/inconsistent expression) are introduced heuristically; a brief explanation of how the Hadamard product separates them would improve clarity.
  3. [Section 4.4, Eq. (10)] The notation \hat{L}(K) in Eq. (10) is inconsistent with \hat{L}^K used in the proof. A superscript convention should be fixed.
  4. [References] Several references contain '?utm_source=chatgpt.com' query strings (e.g., Refs. [1], [5], [7], [37]). These should be removed for a professional reference list.

Circularity Check

0 steps flagged

No significant circularity: the paper's modules are evaluated on external benchmarks and no prediction reduces by construction to a fitted input.

full rationale

The paper's central derivation chain is a conceptual decomposition of hateful memes into two features (presupposed context and false claims), grounded in external philosophical and psychological citations, followed by an ML architecture (PCM, SPM, CRM) tested on independent benchmarks (FHM, Harm-C, Harm-P, ReCOVery, GossipCop). No equation defines a target quantity in terms of itself, and no fitted parameter is renamed as a prediction. The cross-modal reference graph is built from attention scores of an LLM fine-tuned on binary hate labels (Eqs. 5-6), but this is an architectural assumption and a construct-validity concern, not a circular reduction: the graph is not claimed to be a prediction whose validation is its own construction, and the final hateful/non-hateful prediction is evaluated against held-out external labels. Similarly, the proof of Theorem 4.1 does not actually establish sign(Δl) ≠ sign(l), but that is a logical gap in a supporting analysis, not circularity. The only self-citations are non-load-bearing (dataset-statistics notes and a baseline model from the same group). Thus, while the false-claim detection mechanism is under-validated and the theory is not independently confirmed by the model's accuracy, the paper does not exhibit self-definitional, fitted-input-as-prediction, or self-citation-driven circularity.

Axiom & Free-Parameter Ledger

2 free parameters · 6 axioms · 0 invented entities

The framework relies on two philosophical/psychological constructs borrowed from prior literature (presupposed context, false claims), plus a modeling assumption that LLM attention scores faithfully represent cross-modal references. No new physical or conceptual entities with external falsifiable handles are introduced; the cross-modal reference graph is an internal learned representation, not an independently testable entity.

free parameters (2)
  • K (top-K patch neighbors per text token) = 4 or 8 depending on dataset (tuned on validation)
    Eq. 5 defines token-patch edges via top-K attention indices; Table 6 shows optimal K varies by dataset, so K is a per-dataset fitted choice.
  • Text truncation lengths for fake-news adaptation = 150 words (ReCOVery), 50 words (GossipCop)
    Appendix C.2: text is truncated to fit GPU memory; this choice affects fake-news results and is not justified by accuracy criteria.
axioms (6)
  • domain assumption Hate speech presupposes an evaluative context (Marques 2023).
    Core of PCM; if presupposed context is not essential, the module is just generic multimodal fusion.
  • domain assumption Malicious and ideological hate relies on vague or false reasoning (Vendrell 2021).
    Core of FACT; if false, false claims are not a defining feature and SPM/CRM lack theoretical grounding.
  • ad hoc to paper LLM attention after fine-tuning encodes correct token-patch referential links.
    CRM builds graph from top-K attention scores (Eq. 5-6); no evidence attention reflects object-level reference, and the LLM is fine-tuned on binary hate labels, not on reference annotations.
  • ad hoc to paper GCN with linear propagation and mean pooling is representative of the CRM.
    Theorem 4.1 relies on this simplified model; the actual GCN uses nonlinearity and possibly different normalization.
  • domain assumption LLM latent knowledge supplies sufficient societal knowledge for stereotype detection.
    SPM does not use an external knowledge base; it depends on what the fine-tuned LLM learned, and known LLM bias against detecting hate is acknowledged in the paper.
  • standard math Cauchy-Schwarz inequality.
    Used in the derivation of Theorem 4.1, Eq. 13.

pith-pipeline@v1.3.0-alltime-deepseek · 18688 in / 15101 out tokens · 132175 ms · 2026-08-04T10:22:17.128241+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Unpacking Hateful Memes: Presupposed Context and False Claims." pith.science (2026). https://pith.science/paper/KEY4FGYI

@misc{pith2026251009935,
  author       = {Pith},
  title        = {Pith review of: Unpacking Hateful Memes: Presupposed Context and False Claims},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KEY4FGYI}},
  note         = {Machine review of arXiv:2510.09935}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

While memes are often humorous, they are frequently used to disseminate hate, causing serious harm to individuals and society. Current approaches to hateful meme detection mainly rely on pre-trained language models. However, less focus has been dedicated to \textit{what make a meme hateful}. Drawing on insights from philosophy and psychology, we argue that hateful memes are characterized by two essential features: a \textbf{presupposed context} and the expression of \textbf{false claims}. To capture presupposed context, we develop \textbf{PCM} for modeling contextual information across modalities. To detect false claims, we introduce the \textbf{FACT} module, which integrates external knowledge and harnesses cross-modal reference graphs. By combining PCM and FACT, we introduce \textbf{\textsf{SHIELD}}, a hateful meme detection framework designed to capture the fundamental nature of hate. Extensive experiments show that SHIELD outperforms state-of-the-art methods across datasets and metrics, while demonstrating versatility on other tasks, such as fake news detection.

Figures

Figures reproduced from arXiv: 2510.09935 by Jiayu Li, Reza Zafarani, Weibin Cai.

Figure 1
Figure 1. Figure 1: An example of a hateful meme. Meme text: good guy police officer, capturing them young. now often refers to humorous or satirical image-text combinations that convey individual ideologies and rapidly evolve online. However, in some cases, the rapid spread of memes has been exploited to disseminate hate, reinforcing societal biases and threat￾ening social harmony [44]. At the individual level, exposure to h… view at source ↗
Figure 2
Figure 2. Figure 2: Examples illustrating how memes express hate through presupposed context and false claims. Text in Figure 2a and 2b: “good guy police officer, capturing them young.” Blue boxes mark elements reflecting the presupposed context; green boxes mark entities portrayed as “good”, while red boxes denote those portrayed as “bad”; yellow boxes indicate referential links between text and image. In Figure 2a and Figur… view at source ↗
Figure 3
Figure 3. Figure 3: SHIELD Framework. Given a meme image and its text, we first obtain patch and token embeddings 𝐻𝑣 and 𝐻𝑡 from 𝐼𝐸(· ) and 𝑇 𝐸(· ). (1) Input 𝐻𝑣 and 𝐻𝑡 into PCM, where image and text context encoders perform intra-modal interactions to extract modality-specific context embeddings. These are further fused via a context fusion module to produce ℎ𝑃𝐶 , which contain context information of the meme. (2) Meanwhile,… view at source ↗
Figure 4
Figure 4. Figure 4: Examples of two context types of hateful memes. In Fig￾ure 4a, Islam is framed as the bad group. The words “violent” and “kill” are reinforced by the image, which portrays a defiant, con￾frontational crowd——amplifying the negative portrayal through visual alignment. In contrast, in Figure 4b uses positive term “respect” to refer to an execution scene featuring nooses and guillotines. The opposing sentiment… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

74 extracted references · 22 linked inside Pith

  1. [1]

    [n. d.]. France online hate speech law to force social media sites to act quickly. https://www.theguardian.com/world/2019/jul/09/france-online-hate- speech-law-social-media?utm_source=chatgpt.com

    2019

  2. [2]

    [n. d.]. Hate Crimes Case Examples. https://www.justice.gov/hatecrimes/hate- crimes-case-examples?utm_source=chatgpt.com

  3. [3]

    [n. d.]. How the EU Digital Services Act (DSA) Affects Online Free Speech in

  4. [4]

    [n. d.]. Internet giants could be fined up to 12 million under Austrian hate speech law. https://www.reuters.com/article/world/internet-giants-could-be-fined-up- to-12-million-under-austrian-hate-speech-law-idUSKBN25U1R8/

  5. [5]

    [n. d.]. Italian opposition file complaint over far-right party’s use of ‘racist’ AI images. https://www.theguardian.com/technology/2025/apr/18/italian- opposition-complaint-far-right-matteo-salvini-lega-racist-ai-images?utm_ source=chatgpt.com

    2025

  6. [6]

    [n. d.]. Network Enforcement Act. https://en.wikipedia.org/wiki/Network_ Enforcement_Act?utm_source=chatgpt.com

  7. [7]

    [n. d.]. Qualitative research into the impact of online hate. https: //www.ofcom.org.uk/siteassets/resources/documents/research-and- data/online-research/impact-of-online-hate/qualitative-research-into-the- impact-of-online-hate?utm_source=chatgpt.com&v=329170

  8. [8]

    [n. d.]. UK’s new online safety law adds to crackdown on Big Tech com- panies. https://apnews.com/article/online-safety-bill-uk-tech-regulation- 4371bbb0d7442eed0f44bf7839443268

  9. [9]

    The enemy among us

    Wafa Alorainy, Pete Burnap, Han Liu, and Matthew L Williams. 2019. “The enemy among us” detecting cyber hate speech with threats-based othering language Unpacking Hateful Memes: Presupposed Context and False Claims Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Table 4:Model specificity and generalization across hate targets. Harm-C + Harm-P deno...

    2019

  10. [10]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A frontier large vision- language model with versatile abilities.arXiv preprint arXiv:2308.12966(2023)

    Pith/arXiv arXiv 2023

  11. [11]

    Alberto Baldrati, Lorenzo Agnolucci, Marco Bertini, and Alberto Del Bimbo. 2023. Zero-shot composed image retrieval with textual inversion. InProceedings of the IEEE/CVF International Conference on Computer Vision. 15338–15347

    2023

  12. [12]

    Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. Llm2vec: Large language models are secretly powerful text encoders.arXiv preprint arXiv:2404.05961(2024)

    Pith/arXiv arXiv 2024

  13. [13]

    Giovanni Burbi, Alberto Baldrati, Lorenzo Agnolucci, Marco Bertini, and Alberto Del Bimbo. 2023. Mapping memes to words for multimodal hateful meme clas- sification. InProceedings of the IEEE/CVF International Conference on Computer Vision. 2832–2836

    2023

  14. [14]

    Pete Burnap and Matthew L Williams. 2016. Us and them: identifying cyber hate on Twitter across multiple protected characteristics.EPJ Data science5, 1 (2016), 11

    2016

  15. [15]

    Rui Cao, Ming Shan Hee, Adriel Kuek, Wen-Haw Chong, Roy Ka-Wei Lee, and Jing Jiang. 2023. Pro-cap: Leveraging a frozen vision-language model for hateful meme detection. InProceedings of the 31st ACM International Conference on Multimedia. 5244–5252

    2023

  16. [16]

    Rui Cao, Roy Ka-Wei Lee, Wen-Haw Chong, and Jing Jiang. 2023. Prompting for multimodal hateful meme classification.arXiv preprint arXiv:2302.04156(2023)

    Pith/arXiv arXiv 2023

  17. [17]

    Rui Cao, Roy Ka-Wei Lee, and Jing Jiang. 2024. Modularized Networks for Few- shot Hateful Meme Detection. InProceedings of the ACM on Web Conference 2024

    2024

  18. [18]

    Yixuan Chen, Dongsheng Li, Peng Zhang, Jie Sui, Qin Lv, Lu Tun, and Li Shang

  19. [19]

    Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805(2018)

    Pith/arXiv arXiv 2018

  20. [20]

    eu2008 [n. d.]. Framework Decision on combating certain forms and expressions of racism and xenophobia by means of criminal law. https://eur-lex.europa.eu/ legal-content/EN/TXT/?uri=LEGISSUM:l33178

  21. [21]

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. 2022. An image is worth one word: Personalizing text-to-image generation using textual inversion.arXiv preprint arXiv:2208.01618 (2022)

    Pith/arXiv arXiv 2022

  22. [22]

    John Hewitt and Christopher D Manning. 2019. A structural probe for finding syntax in word representations. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 4129–4138

    2019

  23. [23]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685(2021)

    Pith/arXiv arXiv 2021

  24. [24]

    Junhui Ji, Wei Ren, and Usman Naseem. 2023. Identifying creative harmful memes via prompt based approach. InProceedings of the ACM Web Conference

    2023

  25. [25]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation.Comput. Surveys55, 12 (2023), 1–38

    2023

  26. [26]

    Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, and Davide Testuggine. 2020. The hateful memes chal- lenge: Detecting hate speech in multimodal memes.Advances in neural informa- tion processing systems33 (2020), 2611–2624

    2020

  27. [27]

    Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks.arXiv preprint arXiv:1609.02907(2016)

    Pith/arXiv arXiv 2016

  28. [28]

    Gokul Karthik Kumar and Karthik Nandakumar. 2022. Hate-clipper: Multimodal hateful meme classification based on cross-modal interaction of clip features. arXiv preprint arXiv:2210.05916(2022)

    Pith/arXiv arXiv 2022

  29. [29]

    Ruth Lewis, Mike Rowe, and Clare Wiper. 2019. Online/offline continuities: Exploring misogyny and hate in online abuse of feminists.Online othering: Exploring digital violence and discrimination on the Web(2019), 121–143

    2019

  30. [30]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InInternational conference on machine learning. PMLR, 19730–19742

    2023

  31. [31]

    Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang

  32. [32]

    Hongzhan Lin, Ziyang Luo, Wei Gao, Jing Ma, Bo Wang, and Ruichao Yang

  33. [33]

    Phillip Lippe, Nithin Holla, Shantanu Chandra, Santhosh Rajamanickam, Geor- gios Antoniou, Ekaterina Shutova, and Helen Yannakoudakis. 2020. A multimodal framework for the detection of hateful memes.arXiv preprint arXiv:2012.12871 (2020)

    Pith/arXiv arXiv 2020

  34. [34]

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024. Llava-next: Improved reasoning, ocr, and world knowledge

    2024

  35. [35]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024. Visual instruc- tion tuning.Advances in neural information processing systems36 (2024)

    2024

  36. [36]

    Yinhan Liu. 2019. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692(2019)

    Pith/arXiv arXiv 2019

  37. [37]

    Kara Manke. [n. d.]. Study finds persistent spike in hate speech on X. https://news.berkeley.edu/2025/02/13/study-finds-persistent-spike-in-hate- speech-on-x/?utm_source=chatgpt.com

    2025

  38. [38]

    Teresa Marques. 2023. The expression of hate in hate speech.Journal of Applied Philosophy40, 5 (2023), 769–787

    2023

  39. [39]

    Saul Mcleod. 2023. Social identity theory in psychology (Tajfel & Turner, 1979). Simply psychology(2023)

    2023

  40. [40]

    Jingbiao Mei, Jinghong Chen, Weizhe Lin, Bill Byrne, and Marcus Tomalin. 2024. Improving Hateful Meme Detection through Retrieval-Guided Contrastive Learn- ing. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 5333–5347

    2024

  41. [41]

    meta2025 [n. d.]. Hateful Conduct. https://transparency.meta.com/en-us/policies/ community-standards/hateful-conduct/

  42. [42]

    Ron Mokady, Amir Hertz, and Amit H Bermano. 2021. Clipcap: Clip prefix for image captioning.arXiv preprint arXiv:2111.09734(2021)

    Pith/arXiv arXiv 2021

  43. [43]

    Niklas Muennighoff. 2020. Vilio: State-of-the-art visio-linguistic models applied to hateful memes.arXiv preprint arXiv:2012.07788(2020)

    Pith/arXiv arXiv 2020

  44. [44]

    Delfina Sol Martinez Pandiani, Erik Tjong Kim Sang, and Davide Ceolin. 2024. Toxic Memes: A Survey of Computational Perspectives on the Detection and Explanation of Meme Toxicities.arXiv preprint arXiv:2406.07353(2024)

    Pith/arXiv arXiv 2024

  45. [45]

    Agnieszka Pluta, Joanna Mazurek, Jakub Wojciechowski, Tomasz Wolak, Wiktor Soral, and Michał Bilewicz. 2023. Exposure to hate speech deteriorates neurocog- nitive mechanisms of the ability to understand others’ pain.Scientific Reports13, Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Weibin Cai, Jiayu Li, and Reza Zafarani 1 (2023), 4127

    2023

  46. [46]

    Shraman Pramanick, Dimitar Dimitrov, Rituparna Mukherjee, Shivam Sharma, Md Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. 2021. Detecting harmful memes and their targets.arXiv preprint arXiv:2110.00413(2021)

    Pith/arXiv arXiv 2021

  47. [47]

    Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. 2021. MOMENTA: A multimodal framework for detecting harmful memes and their targets.arXiv preprint arXiv:2109.05184 (2021)

    Pith/arXiv arXiv 2021

  48. [48]

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2016. Faster R-CNN: To- wards real-time object detection with region proposal networks.IEEE transactions on pattern analysis and machine intelligence39, 6 (2016), 1137–1149

    2016

  49. [49]

    Koustuv Saha, Eshwar Chandrasekharan, and Munmun De Choudhury. 2019. Prevalence and psychological effects of hateful speech in online college commu- nities. InProceedings of the 10th ACM conference on web science. 255–264

    2019

  50. [50]

    Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dongwon Lee, and Huan Liu

  51. [51]

    Shardul Suryawanshi, Bharathi Raja Chakravarthi, Mihael Arcan, and Paul Buite- laar. 2020. Multimodal meme dataset (MultiOFF) for identifying offensive content in image and text. InProceedings of the second workshop on trolling, aggression and cyberbullying. 32–41

    2020

  52. [52]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971(2023)

    Pith/arXiv arXiv 2023

  53. [53]

    un [n. d.]. Understanding hate speech. https://www.un.org/en/hate-speech/ understanding-hate-speech/what-is-hate-speech

  54. [54]

    Riza Velioglu and Jewgeni Rose. 2020. Detecting hate speech in memes using multimodal deep learning approaches: Prize-winning solution to hateful memes challenge.arXiv preprint arXiv:2012.12975(2020)

    Pith/arXiv arXiv 2020

  55. [55]

    Íngrid Vendrell Ferran. 2021. Hate: Toward a four-types model.Review of Philosophy and Psychology(2021), 1–19

    2021

  56. [56]

    Sebastian Wachs, Manuel Gámez-Guadix, and Michelle F Wright. 2022. Online hate speech victimization and depressive symptoms among adolescents: The protective role of resilience.Cyberpsychology, Behavior, and Social Networking 25, 7 (2022), 416–423

    2022

  57. [57]

    Yang Wu, Pengwei Zhan, Yunjian Zhang, Liming Wang, and Zhen Xu. 2021. Mul- timodal fusion with co-attention networks for fake news detection. InFindings of the association for computational linguistics: ACL-IJCNLP 2021. 2560–2569

    2021

  58. [58]

    x2023 [n. d.]. Hateful Conduct. https://help.x.com/en/rules-and-policies/hateful- conduct-policy

  59. [59]

    youtube2025 [n. d.]. Hate Speech Policy. https://support.google.com/youtube/ answer/2801939

    arXiv

  60. [60]

    Wenwen Yu, Yuliang Liu, Wei Hua, Deqiang Jiang, Bo Ren, and Xiang Bai. 2023. Turning a clip model into a scene text detector. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6978–6988

    2023

  61. [61]

    Weibo Zhang, Guihua Liu, Zhuohua Li, and Fuqing Zhu. 2020. Hateful memes detection via complementary visual and linguistic networks.arXiv preprint arXiv:2012.04977(2020)

    Pith/arXiv arXiv 2020

  62. [62]

    Yuhui Zhang, Alyssa Unell, Xiaohan Wang, Dhruba Ghosh, Yuchang Su, Ludwig Schmidt, and Serena Yeung-Levy. 2024. Why are Visually-Grounded Language Models Bad at Image Classification?arXiv preprint arXiv:2405.18415(2024)

    Pith/arXiv arXiv 2024

  63. [63]

    Xinyi Zhou, Jiayu Li, Qinzhou Li, and Reza Zafarani. 2023. Linguistic-style-aware neural networks for fake news detection.arXiv preprint arXiv:2301.02792(2023)

    Pith/arXiv arXiv 2023

  64. [64]

    Xinyi Zhou, Apurva Mulay, Emilio Ferrara, and Reza Zafarani. 2020. Recovery: A multimodal repository for covid-19 news credibility research. InProceedings of the 29th ACM international conference on information & knowledge management. 3205–3212

    2020

  65. [65]

    Xinyi Zhou, Jindi Wu, and Reza Zafarani. 2020. : Similarity-aware multi-modal fake news detection. InPacific-Asia Conference on knowledge discovery and data mining. Springer, 354–367

    2020

  66. [66]

    Yi Zhou, Zhenhao Chen, and Huiyuan Yang. 2021. Multimodal learning for hateful memes detection. In2021 IEEE International conference on multimedia & expo workshops (ICMEW). IEEE, 1–6

    2021

  67. [67]

    Yangming Zhou, Yuzhou Yang, Qichao Ying, Zhenxing Qian, and Xinpeng Zhang

  68. [68]

    <M> This meme is hateful or not?

    Ron Zhu. 2020. Enhance multimodal transformer with external label and in- domain pretrain: Hateful meme challenge winning solution.arXiv preprint arXiv:2012.08290(2020). A Details of Evaluation In this section, we provide details of baselines and implementation. A.1 Baselines The detailed description of the comparison baseline is as follows: • CLIP Text-O...

    Pith/arXiv arXiv 2020

  69. [2019]

    arXiv preprint arXiv:1908.03557(2019)

    Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557(2019)

    Pith/arXiv arXiv 1908

  70. [2020]

    Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media.Big data8, 3 (2020), 171–188

    2020

  71. [2022]

    In Proceedings of the ACM web conference 2022

    Cross-modal ambiguity learning for multimodal fake news detection. In Proceedings of the ACM web conference 2022. 2897–2905

    2022

  72. [2023]

    In2023 IEEE International Conference on Multimedia and Expo (ICME)

    Multimodal fake news detection via clip-guided learning. In2023 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2825–2830

  73. [2024]

    InProceedings of the ACM on Web Conference

    Towards explainable harmful meme detection through multimodal debate between large language models. InProceedings of the ACM on Web Conference

  74. [2025]

    https://adfinternational.org/commentary/eu-digital-services-act-one-year