Pith. sign in

REVIEW 3 major objections 4 minor 40 references

Modeling text–image mismatch at token, phrase, and global levels outperforms conventional fusion for sarcasm and cyberbullying detection.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 21:24 UTC pith:SB265XX6

load-bearing objection Genuinely new hierarchical incongruity architecture, but the empirical support is undermined by internally inconsistent metrics and a cleaned training split; worth refereeing but not believing at face value. the 3 major comments →

arxiv 2607.16076 v1 pith:SB265XX6 submitted 2026-07-17 cs.CV cs.AIcs.CL

HCIG: A Hierarchical Cross-Modal Incongruity Graph Network for Multimodal Sarcasm and Cyberbullying Detection

Bhavana Verma , Priyanka Meel , Dinesh Kumar Vishwakarma This is my paper
classification cs.CV cs.AIcs.CL
keywords multimodal sarcasm detectioncyberbullying detectioncross-modal incongruityhierarchical graph attention networksimage-text mismatchMMSD benchmarkMultiBully dataset
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that the right way to detect sarcasm and cyberbullying in image–text posts is to model the mismatch between what the text says and what the image shows, at several levels of granularity, rather than simply combining the two modalities. To that end it introduces HCIG, a graph network that scores incongruity between individual words and image patches, between three-word phrases and image patches, and between the overall sentence and image, then lets the model learn how much to trust each level. On the MMSD sarcasm benchmark, HCIG reports 85.74% accuracy and 85.29% macro-F1, outperforming text-only, image-only, and late-fusion baselines; a lighter companion architecture, GCCN, reports the highest macro-F1 on the MultiBully cyberbullying set. A sympathetic reader would care because the results suggest that an explicit incongruity inductive bias—the very cue that makes sarcasm and bullying memes work—can be learned end-to-end and may transfer to other multimodal mismatch problems. One caveat the paper itself flags: the MMSD training set was re-cleaned by dropping 31.9% of training rows with missing images while validation and test were left untouched, so the headline numbers rest on that asymmetric cleaning being harmless.

Core claim

On its own terms, the paper's central claim is that cross-modal incongruity is not a single scalar but a graded phenomenon visible at different linguistic grains, and that a model which measures it at each grain and learns to weight the grains beats models that fuse or attend without an explicit mismatch score. HCIG builds a similarity-thresholded graph between text units (tokens, then three-token phrases) and image patches, propagates over the graph with GATv2, and then—instead of aligning each word to its most similar patch—attends toward the most divergent patches, because those are the regions carrying sarcasm or bullying signal. The three level representations are merged by a learned so

What carries the argument

The load-bearing object is HCIG's three-level incongruity pipeline: token-level and phrase-level graph reasoning over a shared primitive—a cosine-similarity-thresholded graph with learned modality embeddings, propagated by two-layer GATv2—followed by incongruity-guided alignment that weighted-averages the least similar image patches for each text unit; plus a global level comparing pooled CLS vectors directly. The three branch outputs and their scalar incongruity scores are fused by a learned softmax attention gate into a single 3d representation for classification. A complementary architecture, GCCN, applies the same graph primitive once over all tokens plus patches and adds a gated contrad

Load-bearing premise

The MMSD headline result assumes that the cleaned training split—created by deleting 31.9% of training rows with missing images while leaving validation and test untouched (Section 4.1.1)—is representative of the original curated benchmark, so that HCIG's 85.74% accuracy measures the architecture rather than which images happened to be downloadable.

What would settle it

Re-run the full MMSD experiment under a controlled cleaning protocol: train on the original 33,696 pairs with missing images replaced by a constant neutral patch, or drop a matched random 31.9% of validation and test rows, and check whether the 85.74% accuracy and the ~1.4-point gap over late fusion survive. If the gap collapses or accuracy drops substantially, the headline result is an artifact of the asymmetric image cleaning rather than of hierarchical incongruity modeling.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • The reported MMSD scores imply explicit incongruity scoring is worth about 1.4 accuracy points over late fusion (85.74 vs 84.35) and about 2 points over text-only BERT (83.72), suggesting fusion alone leaves cross-modal signal on the table.
  • The ablations imply the hierarchy matters: removing the global level drops MMSD accuracy from 85.74 to 84.65, while a token-only variant retains most of the gain (85.44), so coarse and fine mismatch cues complement each other.
  • The learned level-attention gate is interpretable: the token branch dominates across both classes, offering a per-example explanation of where the model senses mismatch.
  • The cross-task direct-transfer results imply sarcasm and bullying incongruity are task-specific—MMSD-trained models reach only 51–56% accuracy on MultiBully—so bullying detection needs in-domain training rather than a sarcasm-pretrained encoder.
  • GCCN's cheaper recipe—a single graph pass plus contradiction-aware top-k pooling—is competitive, achieving the highest macro-F1 on MultiBully (68.66), which suggests the hierarchical extension is most valuable on the larger MMSD set.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the 'attend to the most divergent patch' trick is a generic inductive bias for any task where mismatch is the signal—out-of-context image detection, fake news, or visual misinformation—so HCIG's alignment module could be lifted into those problems without its full graph scaffolding.
  • Editorial extension: because the phrase branch contributed little, trigram phrase pooling may be too coarse; testing syntactic or dependency-based phrase chunks could reveal whether the hierarchy is genuinely useful or mostly a token-level effect.
  • Editorial extension: if the MMSD result is robust, it should be reproducible on MMSD2.0, the de-biased re-release; a drop there would suggest the model partly captured spurious textual cues rather than true multimodal incongruity.
  • Editorial extension: the gate's strong preference for the token branch could be turned into a cheap diagnostic for human moderators—flag posts where token-level incongruity is high even when the predicted class is non-sarcastic or non-bullying, as potential misses.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes two multimodal architectures for sarcasm and cyberbullying detection: HCIG, which models cross-modal incongruity at token, phrase, and global levels using GATv2 graphs and a learned attention gate, and GCCN, a simpler graph-plus-contradiction-pooling model. Text is encoded with RoBERTa, images with ViT; both models are evaluated on MMSD and MultiBully, with in-domain results, ablations, fine-tuning, and a bidirectional cross-task stress test. The headline claims are HCIG's 85.74% accuracy / 85.29% macro-F1 on MMSD and HCIG's best MultiBully accuracy (69.62%) and bullying-class F1 (74.90%), and the conclusion that hierarchical multi-granularity incongruity modeling is more effective than conventional fusion.

Significance. The architectural idea is coherent and worth testing: explicitly contrasting text tokens, phrases, and global CLS representations against image patches in a graph, rather than fusing pooled features, is a reasonable route to exposing cross-modal incongruity. The paper has several strengths: the mathematical formulation in Sections 3.3–3.5 is precise enough to reimplement, the two models share a common graph primitive, the cleaned-data caveat in Section 4.1.1 is disclosed honestly, and result CSVs are promised. If the reported numbers held, the paper would be a useful benchmark contribution. However, the empirical support is currently unreliable: several tables contain Precision/Recall/F1 triples that cannot arise from the same confusion matrix, and the MMSD training-set cleaning makes the headline results not comparable to prior published results. The central claim of hierarchical superiority therefore is not yet established.

major comments (3)
  1. [Tables 5, 6, and 7 (Section 5.2–5.3)] The reported Precision, Recall, and F1 columns are internally inconsistent. For binary classification, F1 = 2PR/(P+R). Table 5 lists GCCN full as P=84.46, R=85.37, F1=82.37, but 2·84.46·85.37/(84.46+85.37) ≈ 84.91; the same F1 value 82.37 appears in Table 4 with different P/R (78.61/86.51). Table 6 full HCIG lists P=85.04, R=85.66, F1=82.74, whose harmonic mean is ≈85.35. Table 7 shows the same pattern throughout, e.g., HCIG in-domain P=70.08, R=68.20, F1=74.90, whereas the harmonic mean is ≈69.13. Since the F1 column is claimed to be derived from the same experiment as P and R, at least one of these columns is not from the stated run. This matters because the hierarchical-superiority claim rests on the full-vs-token-only gap and the ablations in these tables. The authors must audit the result tables, specify whether F1 is positive-class or macro-averaged, and provide corrected numbers o
  2. [Section 4.1.1 / Table 2] The MMSD cleaning is a serious external-validity threat to the headline result. The paper removes 31.9% of training rows due to missing image files but removes 0% of validation/test rows, so the effective training set is 19,694 pairs with an unknown distribution shift relative to the untouched val/test partitions. The paper acknowledges this, but it does not provide any diagnostics (class balance after removal, image-quality distribution, or comparison of the cleaned training set with the original), nor does it run any prior MMSD method under the same cleaned training split. Therefore the 85.74% accuracy and 85.29% macro-F1 cannot be compared to published MMSD numbers, and the internal model comparisons might be artifacts of which images were downloadable. The authors should either run published baselines on their cleaned split, or evaluate on a split where training and test are processe
  3. [Sections 5.1–5.2, Tables 4–6] The main claim that hierarchical multi-granularity modeling is superior to conventional fusion is supported by very small margins with no uncertainty quantification. The full HCIG vs. token-only gap is 85.74 vs. 85.44 accuracy and 85.29 vs. 85.03 macro-F1; the ablation rows differ by tenths of a percent. No seeds, confidence intervals, or significance tests are reported, so these gaps could be noise. Additionally, Figure 13 and the conclusion themselves state that token-level interactions provide the strongest cues and phrase-level contributions are comparatively little, which weakens the 'hierarchical' component of the claim. The authors should provide variance over multiple runs and, ideally, a paired significance test across the ablations, or temper the hierarchical-superiority conclusion.
minor comments (4)
  1. [Section 5.1, Table 4] The table formatting has run-together values in the GCCN row (e.g., '78.6186.5182.37' and '84.7790.9983.19'); please fix spacing/typesetting.
  2. [Section 3.3, Eq. (5)] The definition of A_uw uses '∨' and a self-loop term; the notation is understandable, but the reader must infer that '∨' is logical OR and that the similarity threshold τ is a hyperparameter. Please state explicitly.
  3. [Section 5.3, Table 7] The F1 column appears to be positive-class F1 while Macro-F1 is given separately. This should be stated in the table caption or text, because without that clarification the reader cannot interpret large discrepancies such as Text-BERT F1=68.43 vs. Macro-F1=63.71.
  4. [Section 5.4] The bidirectional cross-task transfer results are described as 'close to the majority-class baseline', but the majority-class baseline accuracy is never quantified in the text or table. Provide the number or a reference.

Circularity Check

0 steps flagged

No derivation-level circularity; the paper's central claims are empirical benchmark comparisons on public datasets, with only non-load-bearing self-citations in related work.

full rationale

The paper's central claim — that hierarchical multi-granularity incongruity modeling outperforms conventional fusion — is supported by held-out evaluations on the public MMSD and MultiBully benchmarks, not by an equation that defines its output in terms of its input. The model architecture (Eqs. 4–20) is fully specified: graph construction thresholds cosine similarity, GATv2 propagates node states, incongruity scores are computed from learned representations, and the level-attention gate combines those representations. Nothing in these equations assumes the target labels or pre-commits to the reported accuracies/F1 scores. The performance claims are empirical, and the comparison against baselines is an external benchmark, so there is no fitted parameter renamed as a prediction and no prediction that is forced by construction. The self-citations [23,24,26] appear only in Related Work to motivate graph attention as a general mechanism; the HCIG/GCCN designs are independently defined and ablated in Tables 5–6, so the central evidence does not reduce to those citations. Other concerns raised in the manuscript — notably the internally inconsistent Precision/Recall/F1 triples in Tables 5–7 (e.g., Table 6 reports P=85.04, R=85.66, F1=82.74, while 2·85.04·85.66/(85.04+85.66)≈85.35) and the asymmetric MMSD train-split cleaning in Section 4.1.1 — are serious reproducibility and external-validity problems, but they are not circularity: inconsistent metrics do not demonstrate that any result was defined in terms of itself, and an unrepresentative training split is a data-quality issue rather than a derivation loop. Under the stated review rules, no circular step can be exhibited, so the appropriate finding is a low non-circularity score.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The architecture introduces no new physical or ontological entities. It relies on hand-set graph/pooling hyperparameters and several assumptions about embedding geometry and dataset reliability; the most consequential is the unverified representativeness of the cleaned MMSD training split.

free parameters (4)
  • Graph similarity threshold τ = 0.3
    Hand-chosen edge-construction threshold for token/patch graphs (Eq. 5, Table 3); no sensitivity analysis is reported.
  • Contradiction top-k k = 10
    Masked top-k pooling size in GCCN (Eq. 11, Table 3); chosen without reported tuning or sensitivity analysis.
  • Phrase group size = 3
    Number of tokens mean-pooled into each phrase in HCIG (Eq. 13, Table 3); no sensitivity analysis is reported.
  • Learned network weights (GATv2, MLPs, projections) = not specified
    All model parameters are fitted to the training data, which is standard for neural nets, but optimization details (learning rate, GAT heads, hidden widths, dropout) are not reported.
axioms (4)
  • domain assumption Cosine similarity in learned token/image-patch embedding space is a valid measure of cross-modal incongruity.
    Used to define graph edges (Eq. 5), contradiction distance d (Eqs. 9, 14), and global incongruity (Eq. 18).
  • ad hoc to paper The cleaned MMSD training subset (72.6% of raw) is representative enough of the untouched val/test splits for the reported accuracy to be meaningful.
    Section 4.1.1 removes 31.9% of training rows but 0% of val/test rows; no distribution matching or re-weighting is applied.
  • domain assumption The binary labels in MMSD and MultiBully are reliable and the image-text pairings are semantically coherent.
    Ground truth is used without annotation-quality analysis; the cited MMSD2.0 work [8] specifically found mis-annotations and spurious cues in the original MMSD.
  • domain assumption GATv2 propagation over a thresholded similarity graph captures cross-modal incongruity rather than merely feature similarity.
    Central to both architectures; no alternative graph constructions or thresholds are compared.

pith-pipeline@v1.3.0-alltime-deepseek · 13352 in / 14414 out tokens · 121981 ms · 2026-08-01T21:24:24.905843+00:00 · methodology

0 comments
read the original abstract

Multimodal sarcasm and cyberbullying detection remain challenging because the intended meaning often emerges from incongruity between textual and visual information rather than from either modality alone. Existing multimodal approaches primarily rely on feature fusion or cross-modal attention, which may not effectively capture hierarchical semantic inconsistencies across different levels of representation. To address this limitation, this paper proposes HCIG (Hierarchical Cross-modal Incongruity Graph Network), a novel framework that models cross-modal incongruity at token, phrase, and global levels using graph attention networks and adaptively integrates these representations through a learned hierarchical attention mechanism. As a complementary architecture, we also introduce GCCN (Graph-based Cross-modal Contradiction Network), which performs graph-based reasoning using contradiction-aware pooling for efficient multimodal interaction learning. The proposed models are evaluated on the MMSD sarcasm benchmark and the MultiBully cyberbullying dataset, together with comprehensive ablation studies and cross-task transfer experiments. Experimental results demonstrate that HCIG achieves the best performance on MMSD with 85.74% accuracy and 85.29% macro-F1, while GCCN attains the highest macro-F1 (68.66%) on MultiBully and HCIG achieves the highest accuracy (69.62%) and bullying-class F1 (74.90%). The findings demonstrate that hierarchical multi-granularity incongruity modeling provides more effective multimodal reasoning than conventional fusion strategies, offering a robust framework for sarcasm and cyberbullying detection in social media.

Figures

Figures reproduced from arXiv: 2607.16076 by Bhavana Verma, Dinesh Kumar Vishwakarma, Priyanka Meel.

Figure 1
Figure 1. Figure 1: Representative examples from a multimodal cyberbullying dataset highlighting the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Conceptual overview of the shared multimodal input, preprocessing, encoder, cross [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Conceptual architecture of GCCN, showing multimodal encoding, cross-modal graph [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Conceptual architecture of HCIG, showing token-, phrase-, and global-level incon [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: MMSD class distribution by split (train/val/test), sarcastic vs. non-sarcastic. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Label and sentiment distribution in the cleaned MultiBully dataset. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Performance comparison of baseline and proposed models on MMSD. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Confusion matrices of the evaluated MMSD models. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: ROC curves (left) and precision–recall curves (right) for the MMSD models. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Decrease in positive-class F1 and macro-F1 after removing or isolating individual [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: MultiBully in-domain comparison across accuracy, positive-class F1, macro-F1, and [PITH_FULL_IMAGE:figures/full_fig_p015_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Accuracy and macro-F1 for GCCN and HCIG across MMSD in-domain evalua [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: HCIG level-attention weights (token/phrase/global) by label. [PITH_FULL_IMAGE:figures/full_fig_p016_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: shows GCCN’s contradiction score distribution by label. HCIG’s level-attention weights show a strong and consistent preference for the token-level branch across both classes, with the global branch contributing a secondary signal and the phrase-level branch contributing comparatively little, consistent with the ablation finding ( [PITH_FULL_IMAGE:figures/full_fig_p016_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 3 linked inside Pith

  1. [1]

    BERT: Pre-training of deep bidi- rectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidi- rectional transformers for language understanding,” inProceedings of NAACL-HLT 2019, pp. 4171–4186, ACL, 2019

    2019

  2. [2]

    Multi-modal sarcasm detection in Twitter with hierarchical fusion model,

    Y. Cai, H. Cai, and X. Wan, “Multi-modal sarcasm detection in Twitter with hierarchical fusion model,” inProceedings of the 57th Annual Meeting of the Association for Computa- tional Linguistics, pp. 2506–2515, ACL, 2019. 17

    2019

  3. [3]

    Detecting sarcasm in multimodal social platforms,

    R. Schifanella, P. de Juan, J. Tetreault, and L. Cao, “Detecting sarcasm in multimodal social platforms,” inProceedings of the 24th ACM International Conference on Multimedia, pp. 1136–1145, ACM, 2016

    2016

  4. [4]

    Reasoning with multimodal sarcastic tweets via model- ing cross-modality contrast and semantic association,

    N. Xu, Z. Zeng, and W. Mao, “Reasoning with multimodal sarcastic tweets via model- ing cross-modality contrast and semantic association,” inProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 3777–3786, ACL, 2020

    2020

  5. [5]

    Modeling intra and inter-modality incongruity for multi-modal sarcasm detection,

    H. Pan, Z. Lin, P. Fu, Y. Qi, and W. Wang, “Modeling intra and inter-modality incongruity for multi-modal sarcasm detection,” inFindings of the Association for Computational Lin- guistics: EMNLP 2020, pp. 1383–1392, ACL, 2020

    2020

  6. [6]

    Multi-modal sarcasm detection with interactive in-modal and cross-modal graphs,

    B. Liang, C. Lou, X. Li, L. Gui, M. Yang, and R. Xu, “Multi-modal sarcasm detection with interactive in-modal and cross-modal graphs,” inProceedings of the 29th ACM Inter- national Conference on Multimedia, pp. 4707–4715, ACM, 2021

    2021

  7. [7]

    Multi-modal sarcasm detection via cross-modal graph convolutional network,

    B. Liang, C. Lou, X. Li, M. Yang, L. Gui, Y. He, W. Pei, and R. Xu, “Multi-modal sarcasm detection via cross-modal graph convolutional network,” inProceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pp. 1767–1777, ACL, 2022

    2022

  8. [8]

    MMSD2.0: Towards a reliable multi-modal sarcasm detection system,

    L. Qin, S. Huang, Q. Chen, C. Cai, Y. Zhang, B. Liang, W. Che, and R. Xu, “MMSD2.0: Towards a reliable multi-modal sarcasm detection system,” inFindings of the Association for Computational Linguistics: ACL 2023, pp. 10834–10845, ACL, 2023

    2023

  9. [9]

    A human- centered systematic literature review of cyberbullying detection algorithms,

    S. Kim, A. Razi, G. Stringhini, P. J. Wisniewski, and M. De Choudhury, “A human- centered systematic literature review of cyberbullying detection algorithms,”Proceedings of the ACM on Human-Computer Interaction, vol. 5, no. CSCW2, pp. 1–34, 2021

    2021

  10. [10]

    Cyberbullying detection and machine learning: A systematic literature review,

    V. Balakrisnan and M. Kaity, “Cyberbullying detection and machine learning: A systematic literature review,”Artificial Intelligence Review, vol. 56, pp. 1375–1416, 2023

    2023

  11. [11]

    A multitask framework for sentiment, emotion and sarcasm aware cyberbullying detection from multi-modal code-mixed memes,

    K. Maity, P. Jha, S. Saha, and P. Bhattacharyya, “A multitask framework for sentiment, emotion and sarcasm aware cyberbullying detection from multi-modal code-mixed memes,” inProceedings of the 45th International ACM SIGIR Conference on Research and Devel- opment in Information Retrieval, pp. 1739–1749, ACM, 2022

    2022

  12. [12]

    The hateful memes challenge: Detecting hate speech in multimodal memes,

    D. Kiela, H. Firooz, A. Mohan, V. Goswami, A. Singh, P. Ringshia, and D. Testuggine, “The hateful memes challenge: Detecting hate speech in multimodal memes,” inAdvances in Neural Information Processing Systems 33, 2020

    2020

  13. [13]

    Exploring hate speech detection in multimodal publications,

    R. G´ omez, J. Gibert, L. Gomez, and D. Karatzas, “Exploring hate speech detection in multimodal publications,” in2020 IEEE Winter Conference on Applications of Computer Vision, pp. 1459–1467, IEEE, 2020

    2020

  14. [14]

    Disentangling hate in online memes,

    R. K.-W. Lee, R. Cao, Z. Fan, J. Jiang, and W.-H. Chong, “Disentangling hate in online memes,” inProceedings of the 29th ACM International Conference on Multimedia, ACM, 2021

    2021

  15. [15]

    Multimodal hate speech detection via cross- domain knowledge transfer,

    C. Yang, F. Zhu, G. Liu, J. Han, and S. Hu, “Multimodal hate speech detection via cross- domain knowledge transfer,” inProceedings of the 30th ACM International Conference on Multimedia, pp. 4505–4514, ACM, 2022

    2022

  16. [16]

    Deciphering hate: Identifying hate- ful memes and their targets,

    E. Hossain, O. Sharif, M. M. Hoque, and S. M. Preum, “Deciphering hate: Identifying hate- ful memes and their targets,” inProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, pp. 8347–8359, ACL, 2024. 18

    2024

  17. [17]

    Detecting hate speech in multi-modal memes,

    A. Das, J. S. Wahi, and S. Li, “Detecting hate speech in multi-modal memes,”arXiv preprint arXiv:2012.14891, 2020

    Pith/arXiv arXiv 2012

  18. [18]

    A multimodal framework for the detection of hateful memes,

    P. Lippe, N. Holla, S. Chandra, S. Rajamanickam, G. Antoniou, E. Shutova, and H. Yan- nakoudakis, “A multimodal framework for the detection of hateful memes,”arXiv preprint arXiv:2012.12871, 2020

    Pith/arXiv arXiv 2012

  19. [19]

    Hate speech detection and racial bias mit- igation in social media based on BERT model,

    M. Mozafari, R. Farahbakhsh, and N. Crespi, “Hate speech detection and racial bias mit- igation in social media based on BERT model,”PLOS ONE, vol. 15, no. 8, p. e0237861, 2020

    2020

  20. [20]

    How attentive are graph attention networks?,

    S. Brody, U. Alon, and E. Yahav, “How attentive are graph attention networks?,” in International Conference on Learning Representations, 2022

    2022

  21. [21]

    Towards multimodal sarcasm detection (an obviously perfect paper),

    S. Castro, D. Hazarika, V. P´ erez-Rosas, R. Zimmermann, R. Mihalcea, and S. Poria, “Towards multimodal sarcasm detection (an obviously perfect paper),” inProceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4619–4629, ACL, 2019

    2019

  22. [22]

    Cyberbullying detection ap- proaches for Arabic texts: A systematic literature review,

    H. Allwaibed, M. Anbar, S. Manickam, and A. Bintang, “Cyberbullying detection ap- proaches for Arabic texts: A systematic literature review,”Frontiers in Artificial Intelli- gence, 2025

    2025

  23. [23]

    Navigating sentiment analysis through fusion, learning, utterance, and attention methods: An extensive four-fold perspective survey,

    B. Verma, P. Meel, and D. K. Vishwakarma, “Navigating sentiment analysis through fusion, learning, utterance, and attention methods: An extensive four-fold perspective survey,” Engineering Applications of Artificial Intelligence, vol. 156, p. 111195, 2025

    2025

  24. [24]

    SAMER-Net: Speaker-aware multimodal emotion recognition with context-aware attention and graph convolutional networks,

    B. Verma, P. Meel, and D. K. Vishwakarma, “SAMER-Net: Speaker-aware multimodal emotion recognition with context-aware attention and graph convolutional networks,”Ap- plied Intelligence, vol. 56, no. 5, p. 162, 2026

    2026

  25. [25]

    MMGCN: Multimodal fusion via deep graph convolu- tion network for emotion recognition in conversation,

    J. Hu, Y. Liu, J. Zhao, and Q. Jin, “MMGCN: Multimodal fusion via deep graph convolu- tion network for emotion recognition in conversation,” inProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pp. 5666–5675, ACL, 2021

    2021

  26. [26]

    MHAM: A novel framework for multi- modal sentiment analysis in memes,

    B. Verma, P. Meel, and D. K. Vishwakarma, “MHAM: A novel framework for multi- modal sentiment analysis in memes,”Knowledge and Information Systems, vol. 67, no. 11, pp. 10355–10394, 2025

    2025

  27. [27]

    A comprehensive survey on graph neural networks,

    Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu, “A comprehensive survey on graph neural networks,”IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 1, pp. 4–24, 2021

    2021

  28. [28]

    Semi-supervised classification with graph convolutional net- works,

    T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional net- works,” inInternational Conference on Learning Representations, 2017

    2017

  29. [29]

    Graph atten- tion networks,

    P. Veliˇ ckovi´ c, G. Cucurull, A. Casanova, A. Romero, P. Li` o, and Y. Bengio, “Graph atten- tion networks,” inInternational Conference on Learning Representations, 2018

    2018

  30. [30]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” inAdvances in Neural Information Processing Systems 30, pp. 5998–6008, 2017

    2017

  31. [31]

    RoBERTa: A robustly optimized BERT pretraining approach,

    Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “RoBERTa: A robustly optimized BERT pretraining approach,”arXiv preprint arXiv:1907.11692, 2019. 19

    Pith/arXiv arXiv 1907

  32. [32]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. De- hghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” inInternational Confer- ence on Learning Representations, 2021

    2021

  33. [33]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” inInternational Conference on Learning Representations, 2019

    2019

  34. [34]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” inInternational Conference on Learning Representations, 2015

    2015

  35. [35]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervision,” inProceedings of the 38th International Conference on Machine Learning, pp. 8748–8763, 2021

    2021

  36. [36]

    ViLBERT: Pretraining task-agnostic visiolin- guistic representations for vision-and-language tasks,

    J. Lu, D. Batra, D. Parikh, and S. Lee, “ViLBERT: Pretraining task-agnostic visiolin- guistic representations for vision-and-language tasks,” inAdvances in Neural Information Processing Systems 32, 2019

    2019

  37. [37]

    Grad- CAM: Visual explanations from deep networks via gradient-based localization,

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad- CAM: Visual explanations from deep networks via gradient-based localization,” in2017 IEEE International Conference on Computer Vision, pp. 618–626, IEEE, 2017

    2017

  38. [38]

    Sinkhorn distances: Lightspeed computation of optimal transport,

    M. Cuturi, “Sinkhorn distances: Lightspeed computation of optimal transport,” inAd- vances in Neural Information Processing Systems 26, pp. 2292–2300, 2013

    2013

  39. [39]

    Evidential deep learning to quantify classification uncertainty,

    M. Sensoy, L. Kaplan, and M. Kandemir, “Evidential deep learning to quantify classification uncertainty,” inAdvances in Neural Information Processing Systems 31, pp. 3183–3193, 2018

    2018

  40. [40]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778, IEEE, 2016. 20

    2016