Pith. sign in

REVIEW 3 major objections 5 minor 20 references

Vision-language models can describe memes more accurately than they can interpret them.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 01:10 UTC pith:TTXB32FF

load-bearing objection A solid, well-controlled diagnostic benchmark with a stable cross-model finding; the single-reference ground truth worry is real but addressable and does not sink the central claim. the 3 major comments →

arxiv 2607.27798 v1 pith:TTXB32FF submitted 2026-07-30 cs.AI

MemeBench: What LVLMs Miss When Interpreting Culture-Dependent Memes

Weihang Wang , Kainan Tu , Jielei Zhang , Run Yang , Boheng Sheng , Yuchen He , Yu Xie , Pengyu Chen
show 4 more authors
Peiyi Li Huyang Sun Longwen Gao Zhouhui Lian
This is my paper
classification cs.AI
keywords meme interpretationvision-language modelscultural knowledgediagnostic benchmarkVIKRretrieval-augmented generationentity groundingACG culture
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to show that, when a meme's meaning depends on cultural knowledge outside the image, current large vision-language models fail in a systematic and diagnosable way: they see what is pictured more reliably than they know what it means. To make this claim testable, the authors introduce a bilingual benchmark of 1,253 memes whose free-form explanations are scored against a four-part checklist called VIKR (Visual clues, Identity links, Knowledge units, Reasoning). Across 26 models, every single one covers visible content more reliably than the background knowledge it must supply, and even the strongest model retains a 22.6 percentage-point gap between the two. The paper also argues that the same diagnosis can guide repair: an entity-guided retrieval method improves complete interpretations by 3.6–7.4% and, compared with generic web search, repairs more failed explanations and breaks fewer correct ones. A sympathetic reader would care because a single accuracy score hides which part of interpretation is missing, whereas this design localizes the missing component and tests whether evidence can fill it.

Core claim

The central claim is that interpreting culture-dependent memes is not the same task as describing them, and current models are consistently worse on the knowledge side. MemeBench scores open-ended explanations with binary checklists in four layers—visible content, exact named entities, required cultural background, and the reasoning that connects them—and requires all four layers to pass for a complete interpretation. On 1,253 Chinese and English memes, every one of 26 models has higher Visual than Knowledge coverage; the best model reaches 60.3% complete success while covering 92.3% of visual items and only 69.7% of knowledge items. Similar aggregate scores also hide different failure patte

What carries the argument

The load-bearing instrument is the VIKR annotation schema, which converts a free-form meme explanation into four binary checklists: Visual clues (what is literally shown), Identity links (exact entity names and source works), Knowledge units (the cultural background an outsider would need), and Reasoning mechanisms (how the joke works). Success is the product of these four dimensions, so every checklist item must pass for the explanation to count as complete. This schema does the diagnostic work by allowing per-layer coverage scores, state-level analysis across the 16 possible pass/fail combinations, and measurement of which components change when evidence is added. The augmentation baseline

Load-bearing premise

The benchmark assumes each meme has one evaluable interpretation, fixed by a single human-written reference and its VIKR checklist; any meme that plausibly supports more than one reading will count a valid explanation as a failure, and the 150-item human study that found only three material conflicts cannot rule out widespread ambiguity in the remaining items.

What would settle it

Re-score the released model responses with a judging rubric that accepts generic descriptions in the Identity layer (for example, allowing 'a detective boy' instead of requiring 'Conan') while keeping all other rules unchanged; if the Visual–Knowledge gap collapses or disappears, the headline gap is partly an artifact of strict naming requirements rather than a pure knowledge deficit. Alternatively, run a fresh blind study on a large random sample of the full benchmark in which two independent annotators each write their own reference and checklist; if material conflicts occur at a rate far ab

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Single-score leaderboards understate the difference between seeing and understanding: models with nearly identical overall success can fail on different layers, so improvements aimed at one layer may be invisible in an aggregate score.
  • Knowledge coverage, not basic perception, is the current bottleneck: even the strongest evaluated model misses roughly a quarter of required background facts, and every model falls on the same side of the Visual–Knowledge gap.
  • Retrieval evidence improves Identity and Knowledge coverage but consistently lowers Visual coverage, so augmentation must be evaluated per component rather than by net score alone.
  • Entity-guided retrieval is a promising repair strategy: it improves complete interpretations by 3.6–7.4% and, on every controlled model, repairs more failed answers and breaks fewer correct ones than generic web search.
  • The benchmark is far from saturated, with only three models above 40% complete success on 1,253 memes, leaving substantial headroom for culturally grounded interpretation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same four-layer diagnosis could be applied beyond ACG-centered memes to news images, advertisements, or regional humor, which would test whether the Visual–Knowledge gap is a general property of culture-dependent image interpretation rather than an artifact of this corpus.
  • The consistent Visual drop under retrieval hints at evidence crowding: retrieved text may distract a model from image-grounded content. A testable extension would vary prompt and evidence formatting to see whether the Visual loss can be prevented while keeping the Knowledge gain.
  • Because success is defined against a single human-written reference, reported gap sizes may be pessimistic where memes legitimately support multiple interpretations; measuring human agreement on a larger random sample would quantify this ceiling.
  • The repair–damage result suggests a practical rule for when to retrieve: models with stronger baseline Reasoning coverage benefit most from targeted retrieval, so component-level diagnosis could inform adaptive retrieval decisions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces MemeBench, a bilingual benchmark of 1,253 Chinese and English memes with open-ended explanation as the task. Each meme is annotated with a human-written reference and a four-dimension VIKR checklist (Visual, Identity, Knowledge, Reasoning), and a response is scored by whether it satisfies every checklist item in each dimension, with overall Success requiring all four dimensions. The paper evaluates 26 LVLMs and reports a systematic gap between Visual coverage and Knowledge coverage, claiming that every model and even the strongest (Gemini-3.1-Pro) retains at least a 22.6% V–K gap. It then introduces KAR, an entity-guided retrieval baseline using CultureBase, and reports controlled augmentation experiments on four models showing KAR improves Success by 3.6–7.4% while generic retrieval also improves Identity and Knowledge, and that KAR has a better repair/damage balance than Search-CoT. The paper also includes human calibration of the dual-judge evaluation, a reference-independence study, domain/language analyses, and metric sensitivity checks.

Significance. If the results hold, MemeBench is a genuinely useful diagnostic instrument: it turns meme interpretation from a holistic score into component-level coverage, and it demonstrates a closed loop where the diagnosis identifies a Knowledge bottleneck and targeted retrieval partially fills it. The paper has several concrete strengths: the full benchmark is evaluated (n=1,253) rather than a subset; the evaluation protocol uses a dual-judge intersection calibrated to human labels with kappa close to human–human agreement; the augmentations are paired and controlled so that CoT, Search-CoT, and KAR differ by one variable at a time; and the authors release frozen responses, judge outputs, and inference configurations, making every reported number recomputable. The main claims are empirically grounded and the diagnostic separation between Visual and Knowledge is consistent across all 26 models and both judges, which gives confidence that the general phenomenon is not an artifact of a single model choice. The remaining risks are validation and statistical rather than conceptual; the reference-independence check is small, and some headline increments lack uncertainty quantification.

major comments (3)
  1. [§3, Appendix A.2] The reference-independence study is load-bearing for every metric in the paper because Success requires all checklist items to pass, but it covers only 150 of 1,253 items. With 3/150 material conflicts (2.0%), the 95% CI extends to roughly 5.7%, implying up to ~70 contested memes in the full benchmark. If ambiguity concentrates in Knowledge or Identity, the headline Visual–Knowledge gap and the KAR gains could be biased, because a valid alternative reading would mark a model's explanation as failed. Please expand the stratified blind re-explanation study to a larger sample (or provide a targeted analysis of the 1,103 unexamined items), and report the sensitivity of the main V–K gap and KAR gains when potential conflicts are excluded or re-scored. The current 150-item study is too small to rule out systematic ambiguity in the remaining 88% of the benchmark.
  2. [§5, Table 3] The central KAR claim is that KAR raises VIKR Success by 3.6–7.4%. These are point estimates with no confidence intervals or paired significance tests. The smallest gain (Gemini-3.1-Pro: +3.6 points on n=1,253) is borderline in magnitude; an approximate paired comparison suggests the difference may not be clearly significant. The repair/damage ratios are also conditional on different Vanilla subsets, so their comparison requires paired bootstrap or McNemar-type testing. Please report bootstrap CIs for all Success deltas and for the repair/damage ratio differences, and state whether the KAR advantage over Search-CoT is statistically significant in each of the four models. Without this, the second headline claim is not yet quantitatively supported.
  3. [§3, B.1] Gemini-3.1-Pro is both the top evaluated model and one of the two automatic judges. Even though judges are blinded to model identity and the intersection is calibrated to human labels, self-preference cannot be fully ruled out, and the effect could be largest for the very model whose scores anchor the leaderboard and the KAR experiments. I recommend disclosing this overlap explicitly in the main text and reporting a sensitivity analysis in which the main claims (the V–K gap and the KAR/Search-CoT results for the four detailed models) are recomputed using only the GPT-5.1 judge, or on the human-labeled 150-item subset. This would address a correctness risk while preserving the interpretability of the dual-judge intersection.
minor comments (5)
  1. [Table 7] The human-reference row is described as scored on the study subset while Gemini-3.1-Pro's row shows full-benchmark numbers. Please clarify explicitly that both rows are computed on the same 150 study items, or report the model's full-benchmark scores separately. The current presentation invites an apples-to-oranges comparison.
  2. [Appendix B, Tables 9 vs 12] Table 9 excludes items that either judge failed to score, while Table 12 and the main text count unscored responses as failures. Please state the denominator convention in each table header so readers do not interpret the state distributions as the same scoring protocol as the leaderboard.
  3. [§4, Appendix C.1] KAR has several fixed hyperparameters (τ=0.5, K=5, B=5, temperature/top-p) with no sensitivity analysis. Since KAR is a proposed baseline, a small robustness check around τ and K would strengthen the claim that the observed gains are not tied to a single threshold.
  4. [Figure 5] Figure 5 is central to the repair/damage claim but is only described qualitatively. Please add the underlying per-model ratio values or a table so readers can verify the '21 strict increases' claim.
  5. [Abstract / Table 1] The comparison table marks MUnd/MetaGPT as having 'Explanation' and 'Decomposition', but the related-work discussion treats these as distinct benchmarks. Please clarify the acronym and ensure the citation list matches the table entries.

Circularity Check

0 steps flagged

No significant circularity: MemeBench's claims are empirical measurements against human-authored references, not derivations from fitted parameters or self-citations.

full rationale

MemeBench is an empirical benchmark paper: the central claims are measurements of LVLM responses against human-written VIKR checklists (e.g., Success_i = V_i I_i K_i R_i, Eq. 1) and controlled retrieval comparisons. The only equations in the paper (Success definition, 3/4 sensitivity, KAR scoring functions) are operational definitions or pipeline scoring rules; none is fitted to the outcome it later 'predicts.' The KAR result is an experimental intervention: the retrieval conditions are matched except for CultureBase, and the reported gains (3.6–7.4%) are measured against the same pre-frozen references, so they are not guaranteed by construction. The reference-independence study (150 items, 3 conflicts, 0 removals) is a validation weakness because it does not rule out ambiguity in the other 1,103 memes, but that is a ground-truth reliability risk, not a circular derivation. The only notable overlap is that Gemini-3.1-Pro is both a judge and the top evaluated model; the paper blinds judges to model identity, uses dual-judge intersection, and reports human calibration (Cohen's kappa = 0.81), mitigating—though not eliminating—self-scoring bias. No cited uniqueness theorem or author-self-citation carries a load-bearing claim; citations to model papers (InternVL3.5, etc.) are external system documentation. Therefore no circular step is exhibited.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

No free parameters enter the benchmark scoring itself; the layer scores are deterministic from annotations. KAR has three hand-set hyperparameters (tau=0.5, K=5, B=5), which are acknowledged as design choices rather than fitted values. CultureBase (~5,000 entities from Moegirl and KnowYourMeme) is an introduced resource whose coverage is not independently benchmarked; it is a constructed artifact rather than a postulated entity.

free parameters (4)
  • CultureBase similarity threshold tau = 0.5
    Retention threshold for entity grounding in KAR (Section 4, Eq. 3; Appendix C.1); chosen by hand, not tuned to benchmark success, but affects which entities get searched.
  • Top-K CultureBase entities K = 5
    Number of retained entity anchors per meme (Section 4, Appendix C.1); fixed budget choice.
  • Web query budget B = 5
    Shared query budget for both retrieval conditions (Section 4, Appendix C.1); equal across conditions to isolate entity grounding.
  • Decoding temperature / top-p = 1.0 / 0.95 for stochastic models; greedy for local
    Inference configuration in Appendix B.1; affects variance and could influence outcomes, but fixed across conditions.
axioms (4)
  • domain assumption Each meme has a single intended meaning captured by a human-written reference; deviations are errors.
    The VIKR checklists and Success metric are built on one reference per meme (Section 3; Appendix A.2 found 3 material conflicts on 150 items).
  • domain assumption LLM-as-a-judge intersection (Gemini-3.1-Pro and GPT-5.1) approximates human scoring closely enough for ranking.
    Calibrated on 150 items: kappa=0.81 vs human 0.84 (Section 3, Table 6).
  • domain assumption The sampled platforms (Bilibili, Reddit, ImgFlip) represent Chinese and English online-cultural ecosystems.
    Used for cross-language comparisons; the paper itself notes platform and domain confounds (Section 5, Table 11).
  • domain assumption Open-ended explanation is the right unit of evaluation for meme interpretation.
    Standard prompt asks for explanation; the benchmark's diagnosis is defined over free-form text (Section 3).

pith-pipeline@v1.3.0-daily-deepseek · 27462 in / 13125 out tokens · 114492 ms · 2026-08-01T01:10:14.685714+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of MemeBench: What LVLMs Miss When Interpreting Culture-Dependent Memes." pith.science (2026). https://pith.science/paper/TTXB32FF

@misc{pith2026260727798,
  author       = {Pith},
  title        = {Pith review of: MemeBench: What LVLMs Miss When Interpreting Culture-Dependent Memes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TTXB32FF}},
  note         = {Machine review of arXiv:2607.27798}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large vision-language models have improved at describing visual content, but accurate descriptions do not ensure interpretation when meaning depends on knowledge beyond the pixels. Memes expose this gap because they rely on cultural entities, background knowledge, and community conventions. Most meme benchmarks reduce interpretation to labels or holistic scores, obscuring where an explanation breaks down. We introduce MemeBench, a diagnostic benchmark of 1,253 Chinese and English memes with human-written references and quality-controlled VIKR annotations, centered on anime, comics, games, and adjacent online subcultures. Its VIKR schema decomposes explanations into Visual clues, Identity links, Knowledge units, and Reasoning mechanisms. Across 26 LVLMs, every model covers visible content more reliably than the knowledge needed to interpret it, and even the strongest retains a 22.6% Visual-Knowledge gap. To test whether this diagnosis can guide improvement, we introduce KAR, an entity-guided retrieval baseline built on CultureBase. Across four controlled models, KAR raises VIKR Success by 3.6-7.4% and, compared with generic retrieval, repairs more answers and breaks fewer. Yet both retrieval conditions improve Identity and Knowledge while reducing Visual coverage in every comparison. MemeBench reveals whether an interpretation succeeds, what is missing, and whether targeted evidence fills the diagnosed gap.

Figures

Figures reproduced from arXiv: 2607.27798 by Boheng Sheng, Huyang Sun, Jielei Zhang, Kainan Tu, Longwen Gao, Peiyi Li, Pengyu Chen, Run Yang, Weihang Wang, Yuchen He, Yu Xie, Zhouhui Lian.

Figure 1
Figure 1. Figure 1: The task and evaluation of MemeBench. a) meme with reference answer. b) VIKR ground truth and per-dimension checklists. c) In this paired example, the Vanilla response misses the entity and the cultural episode and fails I/K/R; KAR (§4) retrieves both and passes all four. More examples in Appendix E. acters, source works, community events, and conventions. MemeBench references 2,072 distinct entities, most… view at source ↗
Figure 2
Figure 2. Figure 2: Benchmark scope and evaluation granularity. (a) Lan￾guage (inner ring) and domain composition within each language (outer ring); per-cell counts in Appendix A.1. (b) Checklist density by dimension, shown as the share of memes with fewer than three versus at least three items. A meme carries 12.2 items on average; Success requires all items to pass. 3 MemeBench MemeBench evaluates open-ended meme interpreta… view at source ↗
Figure 3
Figure 3. Figure 3: KAR pipeline. LVLM cue and entity-hypothesis extraction, CultureBase grounding, budgeted entity-guided web search, and evidence-grounded explanation. 5 Experiments Setup We evaluate MemeBench along two complementary axes. Closed-book evaluation measures image-only interpretation under the standard prompt, while paired augmentation set￾tings test how explanations for the same items change when reasoning gui… view at source ↗
Figure 5
Figure 5. Figure 5: KAR repair–damage balance. Ratio of the repair rate among Vanilla failures to the damage rate among Vanilla passes under KAR, versus Vanilla Reasoning coverage across 26 models. Blue/gray points fall above/below 1; the dashed line marks equal rates, and labels mark the four detailed models. response-level evidence that VIKR captures distinct changes that a single aggregate score cannot expose. Because Sear… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

20 extracted references · 11 linked inside Pith

  1. [2]

    Multi3Hate: Multimodal, multilingual, and multicultural hate speech detection with vision–language models. InPro- ceedings of the 2025 Conference of the Nations of theAmer- icas Chapter of the Association for Computational Linguis- tics: Human Language Technologies (Volume 1: Long Pa- pers), 9714–9731. Caffagni,D.;Cocchi,F.;Moratelli,N.;Sarto,S.;Cornia,M....

    2025

  2. [7]

    Memecap: A dataset for captioning and interpreting memes. InProc. of EMNLP, 1433–1445. Jiang, X.; Wang, Q.; Wu, J.; He, X.; Xu, Z.; Ma, Y.; Piao, M.;Yang,K.;Zheng,X.;Shimizu,R.;etal.2026. AVMeme Exam:AMultimodalMultilingualMulticulturalBenchmark forLLMs’ContextualandCulturalKnowledgeandThinking. arXiv preprint arXiv:2601.17645. Kasu,S.K.R.;Rehman,M.Z.U.;D...

    arXiv 2026

  3. [9]

    InProceedings of the 2024 Conference on Empir- ical Methods in Natural Language Processing, 5769–5790

    Benchmarking vision language models for cultural under- standing. InProceedings of the 2024 Conference on Empir- ical Methods in Natural Language Processing, 5769–5790. Nguyen, K. P.; Li, T.; Zhou, D. L.; Xiong, G.; Balu, P.; Alahari, N.; Huang, A.; Chauhan, T.; Bala, H.; Guzelordu, E.; et al

    2024

  4. [11]

    arXiv:2601.03267

    OpenAI GPT-5 System Card. arXiv:2601.03267. Tan, B. C. Z.; Zheng, W.; Liu, Z.; Chen, N.; Lee, H.; Choo, K.T.W.;andLee,R.K.-W.2026. Blend-vis:Benchmarking multimodal cultural understanding in vision language mod- els. InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), 4647–...

    Pith/arXiv arXiv 2026

  5. [12]

    arXiv:2506.03569

    MiMo-VL Technical Report. arXiv:2506.03569. Team,K.;Bai,T.;Bai,Y.;Bao,Y.;Cai,S.;Cao,Y.;Charles, Y.; Che, H.; Chen, C.; Chen, G.; et al

    Pith/arXiv arXiv

  6. [13]

    Wang,M.;Ren,K.;Jalan,P.;Ashraf,A.;Vu,T.V.;Seethara- man, R.; Nawaz, S.; and Naseem, U

    Kimi K2.5: Visual Agentic Intelligence.arXiv:2602.02276. Wang,M.;Ren,K.;Jalan,P.;Ashraf,A.;Vu,T.V.;Seethara- man, R.; Nawaz, S.; and Naseem, U

    Pith/arXiv arXiv

  7. [14]

    In ProceedingsoftheACMWebConference2026,9788–9799

    From Native Memes to Global Moderation: Cross-Cultural Evaluation of Vision–Language Models for Hateful Meme Detection. In ProceedingsoftheACMWebConference2026,9788–9799. Wang,W.;Gao,Z.;Gu,L.;Pu,H.;Cui,L.;Wei,X.;Liu,Z.; Jing,L.;Ye,S.;Shao,J.;etal.2025.InternVL3.5:Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency.arXiv:2508....

    Pith/arXiv arXiv 2025

  8. [15]

    arXiv:2506.20670

    Mmsearch-r1: Incentivizing lmms to search. arXiv:2506.20670. Xu, B.; Wang, C.; Chen, X.; Lin, H.; and Xia, F

    Pith/arXiv arXiv

  9. [16]

    Yadav,S.;Tilton,L.;Antoniak,M.;Arnold,T.;Li,J.;Pawar, S

    PUNMEMECN: A Benchmark to Explore Vision- LanguageModels’UnderstandingofChinesePunMemes.In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 18705–18721. Yadav,S.;Tilton,L.;Antoniak,M.;Arnold,T.;Li,J.;Pawar, S. M.; Karamolegkou, A.; Frank, S.; An, Z.; Rostamzadeh, N.;etal.2025.EvaluationofCulturalCompetenceofVision- L...

    Pith/arXiv arXiv 2025

  10. [17]

    Yue, X.; Ni, Y.; Zhang, K.; Zheng, T.; Liu, R.; Zhang, G.; Stevens,S.;Jiang,D.;Ren,W.;Sun,Y.;etal.2024

    Mm-vet: Evaluating large multimodal models for integrated capabilities.arXiv:2308.02490. Yue, X.; Ni, Y.; Zhang, K.; Zheng, T.; Liu, R.; Zhang, G.; Stevens,S.;Jiang,D.;Ren,W.;Sun,Y.;etal.2024. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. InProc. of CVPR, 9556–9567. Zhang, C.; Feng, X.; Bai, Y.; Du, X.; ...

    Pith/arXiv arXiv 2024

  11. [18]

    Zhao,Y.;Zhang,P.;andIgnat,O.2026

    VSearcher:Long-HorizonMultimodalSearchAgentviaRe- inforcement Learning.arXiv:2603.02795. Zhao,Y.;Zhang,P.;andIgnat,O.2026. BeyondTranslation: Cross-Cultural Meme Transcreation with Vision-Language Models.arXiv:2602.02510. Zhao, Z.; Zhang, S.; Zhang, Y.; Zhao, Y.; Zhang, Y.; Wang, Z.; Wang, H.; Zhao, Y.; Liang, B.; Zheng, Y.; et al

    arXiv 2026

  12. [19]

    InProceedingsof the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2007–2017

    Multimodal under- standingofmemeswithfairexplanations. InProceedingsof the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2007–2017. Zhou, S.; Mohammad, S. M.; Plank, B.; and Frassinelli, D. 2026.ICame,ISaw,IExplained:BenchmarkingMultimodal LLMsonFigurativeMeaninginMemes.arXiv:2603.23229. A Benchmark Construction and Human Validation A.1 ...

    arXiv 2007

  13. [20]

    visual": {

    for multilingual cosine-similarity retrieval. TheserecordsprovideentityanchorsfortheKARpipeline (§4);theretrievednames,aliases,andsourceworksarethen used to guide web searches for cultural background. D Responsible Use and Dataset Card D.1 Scope and Responsible Use Researchscope.MemeBenchisACG-centeredbydesign. ACG communities are where meme conventions a...

    1969

  14. [2020]

    of NeurIPS, 33: 9459–9474

    Retrieval-augmented generation for knowledge- intensive nlp tasks.Proc. of NeurIPS, 33: 9459–9474. Li, C.; Li, Z.; Jing, C.; Liu, S.; Shao, W.; Wu, Y.; Luo, P.; Qiao,Y.;andZhang,K.2024. Searchlvlms:Aplug-and-play framework for augmenting large vision-language models by searching up-to-date internet knowledge.Proc. of NeurIPS, 37: 64582–64603. Liu, Y.; Dua...

    2024

  15. [2021]

    InFindings of EMNLP, 4439–4455

    MOMENTA: A mul- timodal framework for detecting harmful memes and their targets. InFindings of EMNLP, 4439–4455. Shahroor, A. E.; Kmainasi, M. B.; Hasnat, A.; Dimitrov, D.; Martino,G.D.S.;Nakov,P.;andAlam,F.2026. MemeLens: Multilingual Multitask VLMs for Memes.arXiv preprint arXiv:2601.12539. Singh, A.; Fry, A.; Perelman, A.; Tart, A.; Ganesh, A.; El- Kis...

    Pith/arXiv arXiv 2026

  16. [2022]

    Chen, Y.; Hu, H.; Luan, Y.; Sun, H.; Changpinyo, S.; Ritter, A.;andChang,M.-W.2023

    Murag: Multimodal retrieval-augmented generator for open questionansweringoverimagesandtext.InProc.ofEMNLP, 5558–5570. Chen, Y.; Hu, H.; Luan, Y.; Sun, H.; Changpinyo, S.; Ritter, A.;andChang,M.-W.2023. Canpre-trainedvisionandlan- guage models answer visual information-seeking questions? InProc. of EMNLP, 14948–14968. Chiu, Y. Y.; Jiang, L.; Lin, B. Y.; P...

    2023

  17. [2023]

    GoogleDeepMind.2026.Gemini3.1ProModelCard.Tech- nical report, Google DeepMind

    Mme: A comprehensive evaluation benchmark for multimodal large language models.arXiv:2306.13394. GoogleDeepMind.2026.Gemini3.1ProModelCard.Tech- nical report, Google DeepMind. https://deepmind.google/ models/model-cards/gemini-3-1-pro/. Hu, H.; Luan, Y.; Chen, Y.; Khandelwal, U.; Joshi, M.; Lee, K.; Toutanova, K.; and Chang, M.-W

    Pith/arXiv arXiv 2026

  18. [2024]

    Chen, W.; Hu, H.; Chen, X.; Verga, P.; and Cohen, W

    BGE M3-Embedding: Multi-Lingual, Multi- Functionality, Multi-Granularity Text Embeddings through Self-Knowledge Distillation.arXiv:2402.03216, 4(5). Chen, W.; Hu, H.; Chen, X.; Verga, P.; and Cohen, W

    Pith/arXiv arXiv

  19. [2025]

    Bai, S.; Cai, Y.; Chen, R.; Chen, K.; Chen, X.; Cheng, Z.; Deng,L.;Ding,W.;Gao,C.;Ge,C.;etal.2025

    Llava- onevision-1.5: Fully open framework for democratized mul- timodal training.arXiv:2509.23661. Bai, S.; Cai, Y.; Chen, R.; Chen, K.; Chen, X.; Cheng, Z.; Deng,L.;Ding,W.;Gao,C.;Ge,C.;etal.2025. Qwen3-VL Technical Report.arXiv:2511.21631. Bui, M. D.; Von Der Wense, K.; and Lauscher, A

    Pith/arXiv arXiv 2025

  20. [2026]

    Fu, C.; Chen, P.; Shen, Y.; Qin, Y.; Zhang, M.; Lin, X.; Yang, J.; Zheng, X.; Li, K.; Sun, X.; et al

    M- QUEST–Meme Question-Understanding Evaluation on Se- mantics and Toxicity.arXiv:2603.03315. Fu, C.; Chen, P.; Shen, Y.; Qin, Y.; Zhang, M.; Lin, X.; Yang, J.; Zheng, X.; Li, K.; Sun, X.; et al

    arXiv