Pith. sign in

REVIEW 6 major objections 5 minor 23 references

Are MLMs Trapped in the Visual Room?

T0 review · 6 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Multimodal models that accurately perceive a scene still misclassify sarcastic intent about 17.1% of the time on images and 55.4% on videos, supporting the paper's Visual Room claim that seeing does not entail understanding.

desk verdict A useful evaluation paradigm and a new dataset, but the headline perception–cognition gap numbers rest on a pilot-fitted, inconsistently reported perception filter that has not been validated independently. read the letter →

arxiv 2505.23272 v2 pith:K3HFUJSJ submitted 2025-05-29 cs.CV

Yazhou Zhang , Chunwang Zou , Qimeng Liu , Lu Rong , Ben Yao , Zheng Lian , Qiuchi Li , Peng Zhang
show 1 more author
Jing Qin
This is my paper
classification cs.CV
keywords multimodallargemodelssarcasmunderstandingperception-cognitiongapVisualRoomargumenttwo-tierevaluationframeworkMMSardatasetnon-literalintentscenedescription
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to prove that a model can describe a scene correctly and still not understand it: in the Visual Room sense, seeing is symbol manipulation, not comprehension. To make the claim testable, it builds a two-tier evaluation with a perception gate that checks whether a model's scene description matches a human-written one, followed by a cognition test of sarcasm polarity on only the samples that passed. On a new author-annotated dataset of 924 images and 100 videos, eight multimodal models pass the perception gate often but still misjudge sarcasm about 17.1% of the time on images and 55.4% on videos, with errors concentrated in context integration, emotion reading, and pragmatic inference. If true, the result matters because benchmark scores that mix perception and cognition overstate how close models are to understanding non-literal human communication.

What carries the argument

The operative device is the perception–cognition factorization $P(y, S|V, T) = P(y|S, V, T)\cdot P(S|V)$, which splits any sample into a perception task (describe the scene) and a cognition task (infer sarcasm given the scene). Perception is gated by $\mathrm{Acc}_P = \mathbb{I}[\mathrm{sim}(\hat{S}, S) \geq \delta]$: the model's generated description must reach a hybrid similarity — half cosine similarity, half judgment by a third-party LLM — against a human-authored scene description, with thresholds $\delta = 0.8$ for images and $0.65$ to $0.7$ for videos. Cognition accuracy $\mathrm{Acc}_C = P(\hat{y} = y \mid \mathrm{Acc}_P = 1)$ is then computed only on gated samples, and the perception–cognition gap $\mathrm{Gap}_{P,C} = P(\hat{y} \neq y \mid \mathrm{Acc}_P = 1)$ is the headline number. The supporting asset is the MMSar dataset, whose sarcasm labels are self-annotated by the original content authors and verified by independent reviewers, with human scene descriptions attached so that the gate and the cognition test use the same reference ground.

What would settle it

Run the same two-tier evaluation with a threshold sweep: recompute the cognition gap with $\delta$ set to 0.2, 0.5, and 1.0, and with the LLM-judge half of the similarity score removed so only cosine similarity gates perception. If the roughly 17.1% image gap and 55.4% video gap shrink toward zero as the gate loosens, or swing widely with the judge, the gap is measuring the gate rather than a fixed cognitive limit; the claim also predicts the gap should not vanish when models are handed the gold scene description verbatim, a condition the paper's own scene-augmented experiments partially test with mixed results.

Watch

Extended reading notes

Core claim

The central claim is that perception and cognition can be empirically separated in multimodal models, and that once separated they diverge. On the MMSar dataset, the strongest models describe scenes with high fidelity (GPT-4V reaches 90.4% and Claude 3.7 Sonnet 91.2% perception accuracy under different prompts), yet on the samples each model perceived correctly, sarcasm polarity is still wrong on average 17.1% of the time for images and 55.4% for videos. The paper reads this residual error as the signature of the Visual Room: a model can enumerate objects, expressions, and layout without grasping the intent those details carry. It further shows that end-to-end accuracy can exceed perception accuracy for some models (LongVU, Chat-UniVi, DeepSeek-VL), which it interprets as classification succeeding on superficial cues rather than grounding, and that error analysis of GPT-4V attributes failures mainly to context misinterpretation (83.2%), emotion misunderstanding (73.2%), and missing commonsense (72.6%).

Load-bearing premise

The whole argument rests on treating 'correct perception' as a model's scene description scoring at least 0.8 (images) or 0.65–0.7 (videos) against a human-written description, with half of that score coming from another AI model's judgment — if that gate mislabels what the models actually saw, the 17.1% and 55.4% gaps are products of the measurement, not evidence of a real limit on understanding.

Editorial extensions

If this is right

  • End-to-end classification scores on sarcasm benchmarks overstate understanding: several models score higher end-to-end than on the perception gate alone, implying they exploit surface cues rather than grounded scene content.
  • Feeding models explicit scene descriptions does not reliably close the gap — stronger models can even get worse (GPT-4V's F1 drops 4.5% under scene-augmented prompting) — so the bottleneck is cognitive integration, not missing perceptual detail.
  • Dynamic video is the harder regime: the average gap reaches 55.4% versus 17.1% for images, pointing to temporal-context integration as a distinct weakness.
  • The framework transfers to humor understanding, where the gap is 11.8% to 12.2% for the two strongest models, so the perception–cognition separation is not peculiar to sarcasm.
  • Because the gap traces to context misinterpretation, emotion misunderstanding, and commonsense gaps in GPT-4V's errors, improvements must target those cognitive capacities rather than perception or prompting alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 17.1% and 55.4% gaps replicate under a threshold sweep, the perception–cognition gate is a diagnostic worth standardizing: any benchmark claiming non-literal understanding should report cognition error conditional on verified perception, since a single scalar accuracy cannot distinguish 'mis-saw' from 'misunderstood'.
  • The gate's dependence on one LLM judge for half the similarity score means the threshold is partly set by the very kind of model under test; swapping the judge or using pure cosine similarity is a direct robustness check the paper does not run, and the claim would be stronger if the gap survives it.
  • The counterfactual minimal edits (a raised hand flipping meaning, a caption swap from 'Best' to 'Worst') suggest a cheap training intervention: contrastive fine-tuning on such minimal pairs should shrink the gap, a prediction the zero-shot-only design leaves untested.
  • The author-self-annotation design implies that older third-party-labeled sarcasm datasets may systematically overstate model failures, because label noise inflates apparent cognition error; re-labeling a slice of those datasets under the same author-verified protocol would quantify that.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 5 minor

Summary. This paper proposes the Visual Room argument, a multi-modal analogue of Searle's Chinese Room, and operationalizes it through a two-tier perception–cognition evaluation framework. The authors introduce MMSar, a Reddit-derived sarcasm dataset with 924 static images and 100 dynamic videos, human-written scene descriptions, and author-provided labels. They evaluate eight MLMs under four prompting protocols, reporting perception accuracy, recognition accuracy conditioned on perception-correct samples, and a perception–cognition gap of 17.1% for images and 55.4% for videos. They conclude that even successful visual perception does not entail understanding of sarcastic intent, and that the gap stems from weaknesses in context integration, emotional reasoning, and pragmatic inference.

Significance. The Visual Room framing is a useful conceptual contribution, and the two-tier evaluation is a sensible way to separate surface-level description from pragmatic inference. The dataset, with self-annotated labels and scene descriptions, could be valuable if released. The paper also includes counterfactual analyses, an ablation, and a transfer experiment to humor, which broaden the scope. The qualitative claim that accurate perception does not guarantee sarcasm understanding is plausible and worth investigating. However, the precise headline numbers are not yet established because of the perception-filter design, the inconsistency in the video threshold, and the subreddit confound; the central finding needs a more rigorous measurement before it can be accepted at face value.

major comments (6)
  1. [Section 3.2 and Section 4.3] The perception threshold for videos is given as delta=0.65 in Section 3.2 but as delta=0.7 in Section 4.3, with no pilot distribution or ablation reported. Because Eq. (5) defines Gap_{P,C} by conditioning on Acc_P=1, the headline 17.1% and 55.4% figures depend directly on this threshold. The authors should reconcile the inconsistency, report the pilot results, and provide a sensitivity analysis over delta.
  2. [Section 3.2, Eq. (3)] The hybrid similarity score uses Claude 3.7 as an LLM judge, and Claude 3.7 Sonnet is one of the eight evaluated models in Table 2. The perception pass set is therefore not independent of at least one model whose cognition gap is measured, which could bias the results in either direction. The authors should validate sim_llm against human judgments, report inter-annotator agreement on the perception filter, and rerun the gap analysis with an independent perception check or with Claude excluded as judge.
  3. [Section 4.5 and Table 3] The video gap is computed on 27–61 perceived samples per model, as shown in Table 3. For LongVU under IO prompting, 34 perceived samples and a gap of 51.8% give a standard error of roughly 8.6 percentage points, so the average video gap of 55.4% is not a stable estimate. The paper should report per-model counts, confidence intervals, and significance tests before drawing conclusions about video-level cognition.
  4. [Section 3.3] Sarcastic posts are drawn from r/sarcasm, r/irony, and r/satire, while non-sarcastic posts come from thematically distinct subreddits such as r/love, r/cute, and r/happy. The two classes therefore differ systematically in textual style, community, and topic, so the reported perception–cognition gap may partly reflect domain shift rather than sarcasm understanding. The authors should provide matched non-sarcastic samples or demonstrate that results are robust to subreddit and topic controls.
  5. [Section 4.5 and Abstract] The claim that MLMs fail 'even with correct perception' overstates what Acc_P=1 means. Acc_P=1 only indicates that the model's generated scene description exceeds a pilot-fitted similarity threshold against one human reference (Eqs. 2–3); it is not an independent verification of correct perception. The text should consistently use 'perception-correct according to the threshold' and should report human validation of the perception filter.
  6. [Reproducibility] The paper introduces MMSar as a central contribution but provides no dataset URL, no code, and no evaluation scripts. Without these, the reported numbers in Tables 2–6 and Fig. 5 cannot be verified, and the framework cannot be applied by others. Please release the dataset and code, or provide full annotation guidelines, model outputs, and evaluation details as supplementary material.
minor comments (5)
  1. [Abstract and Section 1] The abstract says 'eight state-of-the-art MLMs' while the introduction says 'six' and lists six models; Table 2 evaluates eight models, so the counts should be reconciled.
  2. [Section 2.1 and Section 3.2] Section 2.1 contains a typo, 'visual question answeringm', and Section 3.2 says 'Support that' where 'Suppose that' is intended.
  3. [Table 1] Table 1 has '#Avgerage' instead of '#Average' and uses inconsistent capitalization; please proofread the table and its caption.
  4. [Section 4.7] The counterfactual analysis refers to 'Claude 4', but the evaluated model is Claude 3.7 Sonnet; the model name should be corrected.
  5. [Section 4.6] The error analysis reports percentages that sum to more than 100% without stating that the categories are multi-label; please clarify the annotation scheme.

Circularity Check

2 steps flagged · score 3.0 of 10

The perception–cognition gap is defined via a pilot-fitted threshold and a self-included LLM judge, so the headline 17.1% and 55.4% figures are partly determined by the evaluation design rather than being an externally forced measurement.

  1. fitted input called prediction [Sec. 3.2, Eqs. (2)–(5); Sec. 4.3; Sec. 4.5]
    "We set the similarity threshold δ = 0.8 for image samples and δ = 0.65 for video samples, based on the results of our pilot study. ... Hence, to quantify the sarcasm understanding failure even when perception succeeds, we further define the perception–cognition gap: GapP,C = P(ˆy≠y | AccP = 1)."

    The headline gap is exactly Eq. (5) evaluated on the set selected by δ. Because δ is a pilot-fitted parameter (and is reported inconsistently as 0.65 in Sec. 3.2 but 0.7 in Sec. 4.3), the 'even with correct perception' claim is not an externally fixed measurement; it is the failure rate on a conditioning event that the authors chose and fit. No pilot distribution, calibration, or error bars are reported, so the 17.1% and 55.4% magnitudes are artifacts of the threshold to an unknown degree rather than a model-only property.

  2. self definitional [Sec. 3.2, Eq. (3); Sec. 4.1]
    "To ensure evaluation robustness, we employ a hybrid approach combining cosine similarity and third-party LLM (namely Claude 3.7) judgment: sim( ˆS, S) = 1/2 · sim_cos( ˆS, S) + 1/2 · sim_llm( ˆS, S). ... we conduct evaluation experiments on MMSar over 8 SoTA MLMs ... (1) Claude 3.7 Sonnet."

    Claude 3.7 is both the LLM judge that defines sim_llm and one of the eight models whose Acc_P and Gap_P,C are measured. For Claude 3.7, the 'perception-correct' set used as the denominator in Eq. (5) is therefore partly determined by Claude 3.7's own judgments, so the conditioning event is not independent of the model being tested. This makes that model's gap, and hence the averaged headline figures, self-referential by construction.

full rationale

The central construction is not circular in the strictest sense: no equation forces the 17.1% value; it is an empirical conditional frequency computed from human labels and model outputs. The Visual Room argument is a framing device rather than a mathematical derivation, and no load-bearing uniqueness theorem or self-citation chain is used. The two remaining concerns are partial circularity and design dependence. First, the perceived-correct set is defined by a pilot-fitted similarity threshold, so the reported gap is not a model-only measurement; changing δ can change the conditioning set and thus the gap. Second, the similarity judge includes Claude 3.7, which is also an evaluated model, so one of the eight reported gaps is measured on a set that the model itself helps define. These issues affect the robustness and interpretation of the headline numbers but do not force the empirical conclusion by definition. The paper's main observation — that sarcasm understanding lags behind perception under its framework — retains independent content, but the precise quantitative claims should be treated as threshold-dependent and partly self-referential.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the perception threshold and the conditional definition of cognition rather than on a mathematical derivation. The Visual Room is introduced as a conceptual framing, but it is not a falsifiable entity with independent evidence. No new physical or formal entities are postulated.

free parameters (3)
  • Perception threshold delta for images = 0.8
    Set from a pilot study in Sec 3.2. It determines which samples count as correctly perceived and enter the cognition accuracy and gap calculation.
  • Perception threshold delta for videos = 0.65 in Sec 3.2, 0.7 in Sec 4.3
    Set from pilot and manual inspection, but the paper reports two different values. It strongly influences the reported video perception-cognition gap.
  • Hybrid similarity weights = 0.5 cosine, 0.5 LLM judge
    Eq. 3 averages cosine similarity and Claude 3.7 judgment with equal weights, without sensitivity analysis. Perception correctness depends on this hand-chosen weighting.
assumptions (5)
  • domain assumption Self-tagged Reddit posts with weak social supervision and reviewer verification provide reliable sarcasm ground truth.
    Sec 3.3 defines labels via author tags, comments, upvotes, and expert review. Label noise would directly bias all reported accuracies.
  • domain assumption Passing the scene-description similarity threshold is equivalent to genuine visual perception.
    Sec 3.2 Eqs. 2-3 define perception correctness via a fitted threshold, not a ground-truth measure of perceptual understanding.
  • domain assumption Human-written scene descriptions are objective and sufficient references for judging perception.
    Sec 3.3 uses two annotators plus a reviewer, but only a pilot kappa of 0.83 is reported and final descriptions remain subjective.
  • domain assumption Conditioning cognition accuracy on perception-passing samples isolates cognitive ability.
    Sec 3.2 Eq. 4 defines cognition accuracy only on AccP=1 samples. Misspecification of the perception stage contaminates the cognition estimates.
  • ad hoc to paper Failure to infer sarcastic intent despite accurate scene description demonstrates lack of genuine understanding.
    Sec 1 and Sec 4.5 interpret the measured gap as evidence for the Visual Room. This assumes the gap reflects cognitive limits rather than label noise or evaluation artifacts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Are MLMs Trapped in the Visual Room?." pith.science (2026). https://pith.science/paper/K3HFUJSJ

@misc{pith2026250523272,
  author       = {Pith},
  title        = {Pith review of: Are MLMs Trapped in the Visual Room?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K3HFUJSJ}},
  note         = {Machine review of arXiv:2505.23272}
}
read the original abstract

Can multi-modal large models (MLMs) that can ``see'' an image be said to ``understand'' it? Drawing inspiration from Searle's Chinese Room, we propose the \textbf{Visual Room} argument: a system may process and describe every detail of visual inputs by following algorithmic rules, without genuinely comprehending the underlying intention. This dilemma challenges the prevailing assumption that perceptual mastery implies genuine understanding. In implementation, we introduce a two-tier evaluation framework spanning perception and cognition. The perception component evaluates whether MLMs can accurately capture the surface-level details of visual contents, where the cognitive component examines their ability to infer sarcasm polarity. To support this framework, We further introduce a high-quality multi-modal sarcasm dataset comprising both 924 static images and 100 dynamic videos. All sarcasm labels are annotated by the original authors and verified by independent reviewers to ensure clarity and consistency. We evaluate eight state-of-the-art (SoTA) MLMs. Our results highlight three key findings: (1) MLMs demonstrate high accuracy in visual perception; (2) even with correct perception, MLMs exhibit an average error rate of ~17.1\% in sarcasm understanding, revealing a significant gap between seeing and understanding; (3) this gap stems from weaknesses in context integration, emotional reasoning, and pragmatic inference. This work provides empirical grounding for the proposed Visual Room argument and offers a new evaluation paradigm for MLMs.

Figures

Figures reproduced from arXiv: 2505.23272 by the authors.

Figure 1
Figure 1. The proposed Visual Room argument. such as GPT-4V [19], LLaVA 1.5 [9], Qwen 3-VL [2], etc., has emerged. Such MLMs have demonstrated impressive performance on visual perception tasks, e.g., image classification, object detection and segmentation, proving their strong perceptual capabilities. However, seeing ̸= understanding. As John Searle’s Chinese Room thought experiment reveals that symbol manipulation does not e… view at source ↗
Figure 2
Figure 2. The proposed perception-cognition framework. as objects, facial expressions, and scene composition) according to prescribed rules, its “understanding” may remain confined to mechanical symbol processing, never reaching the underlying context or intent. This experiment echoes the current challenge faced by MLMs: excelling in perception, yet struggling with higher-order cognitive tasks. By concretizing this thought ex… view at source ↗
Figure 3
Figure 3. The distribution of sarcasm types, topics, and key visual elements. and sufficient training, scene annotation can be performed reliably by human annotators. Finally, the dataset comprises 1024 high-quality multi-modal samples, includ￾ing 924 image–text pairs and 100 video–text pairs, with 521 sarcastic and 503 non-sarcastic instances, as shown in [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The scene description similarity across different MLMs under IO prompting. 4.4 Cognitive Capability Evaluation Results This experiment evaluates sarcasm recognition on only those samples that each model correctly perceived, as shown in [PITH_FULL_IMAGE:figures/full_fi…
Figure 5
Figure 5. Figure 5: The distribution of gap. 4.5 Perception–Cognition Gap Analysis We report the perception–cognition gap, defined as the failure rate on sarcasm recognition among samples that have already passed the perception stage, as shown in [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Error analysis. ever!” for an image of a blue-screened laptop. This turns the sarcasm into direct criticism, which is not sarcasm. Yet both GPT-4V and Claude 4 still classify the image as sarcastic [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Counterfactual reasoning results. (LongVU, Chat-UniVi) perform poorly in all ablation settings, especially when vision is removed. This reflects weaker visual understanding and highlights their dependency on shallow textual associations. In summary, the results demonst…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

23 extracted references · 5 canonical work pages

  1. [1]

    Advances in neural information processing systems35, 23716– 23736 (2022)

    Alayrac, J.B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al.: Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems35, 23716– 23736 (2022)

    work page 2022

  2. [2]

    5-vl technical report

    Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., et al.: Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923 (2025)

    arXiv 2025

  3. [3]

    arXiv preprint arXiv:1906.01815 (2019)

    Castro, S., Hazarika, D., Pérez-Rosas, V., Zimmermann, R., Mihalcea, R., Poria, S.: Towards multimodal sarcasm detection (an _obviously_ perfect paper). arXiv preprint arXiv:1906.01815 (2019)

    arXiv 2019

  4. [4]

    Advances in Neural Information Processing Systems37, 110805–110853 (2024)

    Cheng, Z., Cheng, Z.Q., He, J.Y., Wang, K., Lin, Y., Lian, Z., Peng, X., Hauptmann, A.: Emotion-llama: Multimodal emotion recognition and reasoning with instruction tuning. Advances in Neural Information Processing Systems37, 110805–110853 (2024)

    work page 2024

  5. [5]

    GLM, T., Zeng, A., Xu, B., Wang, B., Zhang, C., Yin, D., Rojas, D., Feng, G., Zhao, H., Lai, H., Yu, H., Wang, H., Sun, J., Zhang, J., Cheng, J., Gui, J., Tang, J., Zhang, J., Li, J., Zhao, L., Wu, L., Zhong, L., Liu, M., Huang, M., Zhang, P., Zheng, Q., Lu, R., Duan, S., Zhang, S., Cao, S., Yang, S., Tam, W.L., Zhao, W., Liu, X., Xia, X., Zhang, X., Gu, ...

    2024

  6. [6]

    arXiv preprint arXiv:2410.21276 (2024)

    Hurst, A., Lerer, A., Goucher, A.P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., Hayes, A., Radford, A., et al.: Gpt-4o system card. arXiv preprint arXiv:2410.21276 (2024)

    arXiv 2024

  7. [7]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Jin, P., Takanobu, R., Zhang, W., Cao, X., Yuan, L.: Chat-univi: Unified visual representation empowers large language models with image and video understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13700–13710 (2024)

    2024

  8. [8]

    In: International conference on machine learning

    Li, J., Li, D., Savarese, S., Hoi, S.: Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. In: International conference on machine learning. pp. 19730–19742. PMLR (2023)

    2023

Show all 23 references
  1. [9]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Liu, H., Li, C., Li, Y., Lee, Y.J.: Improved baselines with visual instruction tuning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 26296–26306 (2024)

    2024

  2. [10]

    arXiv preprint arXiv:2110.08387 (2021) Are MLMs Trapped in the Visual Room? 19

    Liu, J., Liu, A., Lu, X., Welleck, S., West, P., Bras, R.L., Choi, Y., Hajishirzi, H.: Generated knowledge prompting for commonsense reasoning. arXiv preprint arXiv:2110.08387 (2021) Are MLMs Trapped in the Visual Room? 19

    2021 arXiv

  3. [11]

    In: Proceedings of the 32nd ACM International Conference on Multimedia

    Lu, F., Wang, W., Luo, Y., Zhu, Z., Sun, Q., Xu, B., Shi, H., Gao, S., Li, Q., Song, Y., et al.: Miko: Multimodal intention knowledge distillation from large language models for social-media commonsense discovery. In: Proceedings of the 32nd ACM International Conference on Mul...

    2024

  4. [12]

    arXiv preprint arXiv:2403.05525 (2024)

    Lu, H., Liu, W., Zhang, B., Wang, B., Dong, K., Liu, B., Sun, J., Ren, T., Li, Z., Yang, H., et al.: Deepseek-vl: towards real-world vision-language understanding. arXiv preprint arXiv:2403.05525 (2024)

    2024 arXiv

  5. [13]

    0: Towards a reliable multi-modal sarcasm detection system

    Qin, L., Huang, S., Chen, Q., Cai, C., Zhang, Y., Liang, B., Che, W., Xu, R.: Mmsd2. 0: Towards a reliable multi-modal sarcasm detection system. arXiv preprint arXiv:2307.07135 (2023)

    2023 arXiv

  6. [14]

    In: International conference on machine learning

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PmLR (2021)

    2021

  7. [15]

    Scholarpedia4(8), 3100 (2009)

    Searle, J.: Chinese room argument. Scholarpedia4(8), 3100 (2009)

    2009

  8. [16]

    arXiv preprint arXiv:2410.17434 (2024)

    Shen, X., Xiong, Y., Zhao, C., Wu, L., Chen, J., Zhu, C., Liu, Z., Xiao, F., Varadarajan, B., Bordes, F., et al.: Longvu: Spatiotemporal adaptive compression for long video-language understanding. arXiv preprint arXiv:2410.17434 (2024)

    2024 arXiv

  9. [17]

    Information Fusion 103, 102132 (2024)

    Wang, J., Yang, Y., Jiang, Y., Ma, M., Xie, Z., Li, T.: Cross-modal incongruity aligning and collaborating for multi-modal sarcasm detection. Information Fusion 103, 102132 (2024)

    2024

  10. [18]

    arXiv preprint arXiv:2409.12191 (2024)

    Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., et al.: Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191 (2024)

    2024 arXiv

  11. [19]

    arXiv preprint arXiv:2310.09909 (2023)

    Wu, C., Lei, J., Zheng, Q., Zhao, W., Lin, W., Zhang, X., Zhou, X., Zhao, Z., Zhang, Y., Wang, Y., et al.: Can gpt-4v (ision) serve medical applications? case studies on gpt-4v for multimodal medical diagnosis. arXiv preprint arXiv:2310.09909 (2023)

    2023 arXiv

  12. [20]

    arXiv preprint arXiv:2310.11374 (2023)

    Zhang, Y., Wang, M., Wu, Y., Tiwari, P., Li, Q., Wang, B., Qin, J.: Dialoguellm: Context and emotion knowledge-tuned large language models for emotion recogni- tion in conversations. arXiv preprint arXiv:2310.11374 (2023)

    2023 arXiv

  13. [21]

    Advances in Neural Information Processing Systems36, 18794–18805 (2023)

    Zhang, Y., Yu, Y., Guo, Q., Wang, B., Zhao, D., Uprety, S., Song, D., Li, Q., Qin, J.: Cmma: benchmarking multi-affection detection in chinese multi-modal conversations. Advances in Neural Information Processing Systems36, 18794–18805 (2023)

    2023

  14. [22]

    Zhang, Z., Zhang, A., Li, M., Zhao, H., Karypis, G., Smola, A.: Multimodal chain-of- thought reasoning in language models (2024),https://arxiv.org/abs/2302.00923

    2024 arXiv

  15. [23]

    arXiv preprint arXiv:2304.10592 (2023)

    Zhu, D., Chen, J., Shen, X., Li, X., Elhoseiny, M.: Minigpt-4: Enhancing vision- language understanding with advanced large language models. arXiv preprint arXiv:2304.10592 (2023)

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.