Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Token Activation Map to Visually Explain Multimodal LLMs

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Subtracting earlier tokens sharpens multimodal AI explanations

desk verdict TAM is a genuinely new idea for MLLM explainability—modeling context-token interference—but the headline numbers rest on self-defined metrics that may reward suppression, so the quantitative claim needs strengthening before it is fully established. read the letter →

arxiv 2506.23270 v1 pith:3A2QOWGF submitted 2025-06-29 cs.CV cs.AIcs.LG

Yi Li , Hualiang Wang , Xinpeng Ding , Haonan Wang , Xiaomeng Li This is my paper
classification cs.CVcs.AIcs.LG
keywords multimodallargelanguagemodelsvisualexplainabilitytokenactivationmapestimatedcausalinferencerankGaussianfiltercontext-tokeninterferencegroundingIoU-basedevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multimodal large language models generate text one token at a time, and earlier tokens (the context) leak visual activation into later tokens, so a plain class-activation map for a word like "fork" lights up wherever the model already mentioned "plate." The paper claims this interference is widespread and that it degrades the reliability of visual explanations for MLLMs. To address it, the authors propose Token Activation Map (TAM): for each generated token, estimate an interference map from all earlier context tokens, scale it by a least-squares factor, subtract it from the raw activation map, and denoise the result with a rank Gaussian filter. They report F1-IoU gains over existing explanation methods on COCO Caption, GranDf, and OpenPSG, and show the same subtraction improves other methods when applied on top of them. If correct, TAM gives MLLMs a backprop-free way to produce token-level visual explanations for localization, failure analysis, and model comparison.

What carries the argument

The load-bearing object is the estimated causal inference operator $E$ combined with the rank Gaussian filter $D$. $E$ (Eq. 4) computes the interference map as $\sum_{k=1}^{n_p+i-1} \frac{r_k^i}{P r_i + \epsilon} A_k$, where $r_k^i$ is the textual relevance between the k-th context token and the current answer token, obtained from the text-token classifier weights; the relevance of an identical token is set to zero to avoid canceling genuine signal. The scalar $s$ is fitted by least squares (Eq. 5) to make the subtraction comparable in scale. $D$ (Eq. 6) then sorts local values in each sliding window and applies a 1-d Gaussian kernel centered at the median rank, with the kernel width controlled by the coefficient of variation $\sigma/\mu$, so it acts as a smoothed median filter that preserves smaller responses. Together these define TAM in Eq. 2, and the final multimodal activation map concatenates the refined visual map with the textual relevance (Eq. 3).

What would settle it

Take images where a later target token and an earlier context token refer to the same visible object (e.g., a caption "a fork on a plate" and then explain the token "fork" while "plate" is in context). If TAM's Obj-IoU for such co-referent pairs is systematically lower than for pairs where context and target refer to disjoint objects, the subtraction is removing shared signal rather than interference. This can be computed directly on COCO Caption by splitting tokens according to whether their ground-truth masks overlap with masks of context-token referents.

Watch

Extended reading notes

Core claim

The central claim is that the correct explanation of a token in an MLLM is the raw classifier-weight activation map minus the visual activation contributed by earlier context tokens. In the paper's notation, the refined map for the i-th answer token is $\bar{A}^a_i = \lfloor D(A^a_i - s E(A_{:n_p+i-1})) \rfloor_+$, where $E$ builds an interference map as a textual-relevance-weighted sum of all earlier activation maps, $s$ is a scalar fitted by least squares, and $D$ is the proposed rank Gaussian filter that suppresses salt-and-pepper noise. The authors argue that existing explainability tools overlook this context-token interference because they were designed for single-output models, and that removing it substantially improves both object localization and the suppression of spurious activations on function words. They further claim that TAM is complementary to existing methods such as CAM and Grad-CAM++, and that it scales across model families and sizes, supporting video, multi-turn conversation, and multi-image scenarios.

Load-bearing premise

The method assumes that the interference from earlier context tokens is a linear, additive combination of their activation maps, weighted only by textual relevance, so subtracting a scaled estimate removes interference rather than genuine signal; if earlier and later tokens look at the same object, this subtraction removes real evidence.

Editorial extensions

If this is right

  • Token-level heatmaps for every generated word, including function words, become substantially cleaner: object words light up their referents and function words stay close to background.
  • The subtraction can be layered on top of existing CAM, Grad-CAM++, and Layer-CAM to raise their F1-IoU, so TAM behaves as a plug-in rather than a full replacement.
  • The method works across model families (Qwen2-VL, InternVL, LLaVA) and model sizes, suggesting explainability can scale alongside model development.
  • TAM enables qualitative comparison of MLLMs: where a model looks can be compared visually, and failure cases on datasets like QK-VQA can be diagnosed as perception failures versus knowledge misalignment.
  • Repeating the per-token processing over frames and conversation turns extends the method to video visualization and multi-turn dialogue.
  • The paper itself states a limitation: only visual inputs are addressed; audio and other modalities remain unexplored.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Not claimed by the paper: the linearity assumption of the interference subtraction predicts a testable side effect — if a later token and an earlier context token legitimately share a visual region (e.g., "fork" and "plate" both on a table), TAM will suppress that shared region, so its Obj-IoU should dip specifically on co-referent pairs that share ground-truth masks.
  • A natural extension the authors do not explore is fitting per-region or per-context-token scale factors instead of one global $s$, which would soften the linearity assumption and might recover genuine shared signal.
  • Because TAM needs only the token classifier weights and no gradients, it could in principle apply to any autoregressive multimodal model with a linear head, including closed models if logits are exposed; the paper does not claim this.
  • The motivation statistic (higher text correlation corresponds to lower L1 distance between activation maps) suggests that interference scales with semantic similarity, which could be used to build a calibration curve for when the subtraction is most needed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Token Activation Map (TAM), a post-processing method for explaining token predictions of multimodal large language models (MLLMs). TAM computes raw activation maps from the classifier weights, subtracts a scaled 'interference map' estimated from earlier context tokens (Eqs. 2, 4, 5), and applies a novel rank Gaussian filter (Eqs. 6, 7) for denoising. The evaluation introduces three new IoU-based metrics (Obj-IoU, Func-IoU, F1-IoU) and reports consistent F1-IoU gains over existing baselines on COCO Caption, GranDf, and OpenPSG, while also showing that TAM can improve existing CAM-style methods when applied on top of them. The paper further provides extensive qualitative demonstrations, including object localization, attribute analysis, failure-case analysis, video visualization, and MLLM comparison.

Significance. If the quantitative and qualitative claims are accepted, TAM is a useful and broadly applicable tool for MLLM explainability. A particular strength is that the method is lightweight: it requires no back-propagation or attention weights (except for some baselines), works with FlashAttention, and can be applied on top of existing CAM-like methods. The paper presents extensive qualitative evidence across seven MLLMs and several datasets, and the proposed rank Gaussian filter is a simple and interpretable contribution. However, the headline numbers rest on newly proposed metrics whose validity is not independently established, and the per-image least-squares fitting of the scale factor raises concerns about the fairness of the experimental comparison. These issues prevent the current manuscript from being a fully convincing demonstration of state-of-the-art performance.

major comments (4)
  1. [3.4, Eq. (9)] The Func-IoU metric may reward activation suppression rather than explanation quality. Because the background threshold b_i is derived from the mean OTSU threshold of noun-token maps in the same image, any method that systematically lowers activation intensities (as TAM's Eq. 2 does by subtracting a scaled interference map) will trivially increase Func-IoU. The calibration described in Supp. E (replacing the first generated token with the first prompt token's map) affects only one token and does not remove this systematic bias. The paper should validate Func-IoU against human judgments or an independent faithfulness measure, or otherwise show that the metric's behavior is not dominated by the intensity shift.
  2. [3.2, Eq. (5)] The scale factor s is a per-image parameter fitted by least squares on the same image that is later evaluated. While this is not label-leaking, it is an optimized subtraction rather than a parameter-free mechanism, and it puts TAM on a different footing from the baselines, which have no such per-example adaptation. The paper should report the distribution of fitted s values, analyze its sensitivity, and ideally include a version with a single fixed s cross-validated on a small development set to show that the gains are not due to per-image overfitting of this parameter.
  3. [3.2, Eq. (4) and Supp. F] The core modeling assumption that context interference is additive, linear in the raw activation maps, and weighted by textual relevance r_k^i is validated only indirectly through the proposed IoU metrics. This creates a circularity risk: the same metrics are used to motivate the method and to evaluate it. An independent test (e.g., transfer of the produced maps to a weakly-supervised object localization benchmark, or a human subject study) is needed to confirm that the subtraction removes interference rather than genuine target signal.
  4. [4.2, Table 3 and Supp. E] The central claim of state-of-the-art performance rests on F1-IoU, a newly proposed metric. The paper explicitly rejects perturbation-based faithfulness tests and offers no alternative validation, such as a human study or a correlation with standard localization benchmarks. Consequently, the reported gains of 8.96% on COCO Caption, 2.82% on GranDf, and 8.54% on OpenPSG are not yet established as improvements in explanation quality. Please provide at least one independent evaluation, for example by using the resulting maps in a downstream task with a standard metric.
minor comments (5)
  1. [Fig. 2 caption] The caption uses the label '(d)' twice, once for the rank Gaussian filter and once for the evaluation metrics, which makes the figure confusing. The labels should be renumbered so that each panel has a unique identifier.
  2. [3.1 and 2] There are several typos: 'dipect' should be 'depict' (Sec. 3.1), 'involing' should be 'involving' (Related Work), and 'obversely' should be 'obviously' (Sec. 2). These are minor but should be corrected in revision.
  3. [Supp. E] The definitions of plausibility and faithfulness are confusingly worded: both are described as 'how accurately it reflects the true reasoning process'. The distinction between the two concepts should be clarified and stated in terms of the measured property (e.g., agreement with human intuition vs. sensitivity to input perturbation).
  4. [4.1] The description of the OpenPSG mask source ('masks are integrated by Rasheed et al.') is ambiguous. It should be explicitly stated whether these masks are manually annotated or automatically generated, and how they relate to the original dataset annotations.
  5. [Eq. (3)] The multimodal activation map concatenates a visual map of length nv with a textual relevance vector of length np+i-1, but the normalization N(.) used to align the two modalities is not defined in enough detail to be reproducible. Please specify the exact normalization procedure and how the two components are combined in the visualization.

Circularity Check

1 steps flagged · score 5.0 of 10

Partial circularity: the per-image least-squares scale factor and the Func-IoU criterion together build the headline suppression gain; Obj-IoU and the placebo test provide only partial independent support.

  1. fitted input called prediction [Sec. 3.2 (Eq. 5), Sec. 3.4 (Eq. 9), Supp. E]
    "s = arg min_s \sum_{j=1}^{n_v} (A^a_{j,i} - s E(A_{j,:}^{n_p+i-1}))^2. // Eq. 9: Activation map lower than it (A_i < b_i) is the background prediction used to count IoU with the all-one matrix J. // Supp. E: our ECI involves a subtraction operation between activation maps, which can lead to a lower overall intensity compared to the original responses."

    Eq. 2 outputs the positive part of A_i - sE; Eq. 5 fits s on the same image by least-squares residual between A_i and sE. Since E is a weighted sum of nonnegative context activation maps, the fitted operation is a per-image suppression of the very map used to fit it. Eq. 9 defines high Func-IoU as the map falling below the b_i threshold, i.e., as suppression, and Supp. E concedes that subtraction lowers overall intensity and patches thresholds by substituting the first prompt token's unprocessed map. The large Func-IoU gains (Table 1: 51.93 to 69.03) are therefore substantially produced by the fitted subtraction plus a self-defined background criterion, rather than by an independent faithfulness check.

full rationale

The central method is not a fitted predictor of a held-out target; it is an explanation procedure computed from the model's own features, and I find no load-bearing self-citation chain and no uniqueness theorem imported from the authors' prior work. However, the quantitative superiority claim is not fully independent: the Func-IoU metric defines low activation on function words as correct, and Eqs. 2 and 5 perform a per-image least-squares-fitted subtraction that suppresses activation intensity; Supp. E explicitly adjusts the evaluation because of this intensity shift. That makes the Func-IoU component of the headline F1-IoU gain partly circular with the metric. The Obj-IoU component uses external manual masks and passes some internal checks (placebo test, ablations), so the circularity is partial rather than total. Concerns about the lack of human validation and about dismissing perturbation-based faithfulness are correctness or validity risks, not circularity, and should be assessed separately.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical or architectural entities are introduced. The free parameters are the scale factor s, the kernel size k, and the relevance weighting scheme. The axioms are the linearity of interference, the textual-relevance proxy, the validity of the proposed IoU metrics, and the CAM-style weight interpretation.

free parameters (3)
  • scale factor s = Optimized per token via least squares (Eq. 5)
    The scale factor controls how much interference is subtracted from the target activation map. It is fit on the same image and token that is later evaluated, so it is a free parameter that absorbs the residual between the raw map and the interference estimate.
  • rank Gaussian filter kernel size k = 3
    The kernel size is chosen by hand as 'suitable' based on the observation that salt-and-pepper noises cluster between 1 to 5 tokens. It is not derived from first principles.
  • textual relevance weights r_k^i = Computed from classifier activations between text tokens (Eq. 4)
    These weights determine how much each context token contributes to the interference map. They are not learned or optimized, but they are a modeling choice that could be replaced by other similarity measures.
assumptions (4)
  • domain assumption The interference of context tokens on the explained token manifests as an additive linear combination of the context tokens' visual activation maps.
    This is the core assumption behind Eq. 2 and Eq. 4, and it is not independently validated.
  • domain assumption Textual relevance between tokens is a valid proxy for the degree of visual activation interference between those tokens.
    Eq. 4 uses textual relevance r_k^i as the weight for subtracting context activation maps. The paper provides a correlational analysis (Fig. 1c) but no causal validation of this specific proxy.
  • domain assumption The evaluation metrics Obj-IoU, Func-IoU, and F1-IoU are valid proxies for explanation quality.
    The paper proposes these metrics and explicitly avoids faithfulness testing, so the entire quantitative comparison rests on this plausibility assumption.
  • domain assumption Activation maps can be computed from the token classifier weights in the MLLM (Eq. 1), treating the classifier as a linear probe.
    The method inherits the CAM assumption that classifier weights applied to visual features produce meaningful localization maps.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Token Activation Map to Visually Explain Multimodal LLMs." pith.science (2026). https://pith.science/paper/3A2QOWGF

@misc{pith2026250623270,
  author       = {Pith},
  title        = {Pith review of: Token Activation Map to Visually Explain Multimodal LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3A2QOWGF}},
  note         = {Machine review of arXiv:2506.23270}
}
read the original abstract

Multimodal large language models (MLLMs) are broadly empowering various fields. Despite their advancements, the explainability of MLLMs remains less explored, hindering deeper understanding, model credibility, and effective visualization. Unlike conventional vision models (e.g., CNNs, ViTs, CLIP) that produce a single output, MLLMs generate sequences of tokens progressively, where each generated token depends on the previous context. Therefore, earlier context tokens can introduce redundant activations that interfere with the explanation of later tokens beyond their original information. Existing studies often overlook this issue, but our observations reveal that these redundant correlations can significantly hurt the reliability of explanations. To address this, we propose an estimated causal inference method to mitigate the interference of context to achieve high-quality MLLM explanation, with a novel rank Gaussian filter to further reduce activation noises. We term this method Token Activation Map (TAM) to highlight the consideration of interactions between tokens. TAM also indicates that it excels at explaining multiple tokens of MLLM, which is different from the Class Activation Map (CAM) for a single prediction. Our TAM method significantly outperforms existing SoTA methods, showcasing high-quality visualization results that can be utilized for various scenarios, such as object localization, failure case analysis, video visualization, MLLMs visual comparison, and model understanding (e.g., color, shape, action, location, visual reasoning, multi-turn conversation, etc). The code is available atgithub.com/xmed-lab/TAM.

Figures

Figures reproduced from arXiv: 2506.23270 by the authors.

Figure 1
Figure 1. Illustration of Motivation. (a) MLLMs generate multiple tokens beyond a single output. (b) The information flow indicates that MLLM generates tokens progressively, where one generated token (each row) is correlated with the context (prompt + earlier answer tokens). (c) We randomly pair CAMs and count their L1 distance against text correlation. Higher text correlation corre￾sponds to lower distance, indicating concur… view at source ↗
Figure 2
Figure 2. Illustration of method. (a, Eq. 1 - Eq. 3) The overall framework of TAM. (b, Eq. 4 - Eq. 5) Details of the estimated casual inference module. (d, Eq. 6 - Eq. 7) Details of the rank Gaussian filter module. (d, Eq. 8 - Eq. 10) Fine-grained evaluation metrics. be more specific, causal inference is employed to investigate the correct causal relation between the current answer token and visual tokens, eliminating interfe… view at source ↗
Figure 3
Figure 3. Visual comparison between TAM and SoTA methods on the COCO Caption dataset [ [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (27 more)
Figure 4
Figure 4. Figure 4: TAM presents high-quality localization results on di [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: TAM significantly improves the quality of video visualization, with much fewer redundant activation and noises on STAR [ [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: TAM supports failure cases analysis on QK-VQA [ [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: TAM enables qualitative comparison among MLLMs. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: A high-resolution example of the multimodal activation [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Visualization of one example for all generated tokens on the Qwen2-VL-2B [ [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Relation between model size and explainability. ( [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 19
Figure 19. Figure 19: TAM supports analyzing biased scenarios. The [PITH_FULL_IMAGE:figures/full_fig_p018_19.png]
Figure 12
Figure 12. Figure 12: Visual comparison between our TAM and SoTA methods on the COCO Caption dataset [ [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Visual comparison between our TAM and SoTA methods on the COCO Caption dataset [ [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]
Figure 14
Figure 14. Figure 14: The proposed estimated causal inference is well-designed beyond other methods and implementations. The yellow boxes indicate correlated activations, and the blue boxes mean missed activations. Feature surgery [32] is designed for CLIP [41] to mitigate redundant featur…
Figure 15
Figure 15. Figure 15: The proposed rank Gaussian filter is more effective than existing methods. The yellow boxes indicate insufficient denoising, and the blue boxes mean over-denosing. The used model is Qwen2-VL-2B [51] on the COCO Caption dataset [13]. 23 [PITH_FULL_IMAGE:figures/full_f…
Figure 16
Figure 16. Figure 16: TAM involves two mutually beneficial modules. (ECI only) The estimated causal inference can mitigate most of the interference activations, while some stubborn noise remains. (RGF only) The rank Gaussian filter can remove these noises, while redundant activations are o…
Figure 17
Figure 17. Figure 17: TAM supports visual comparison among MLLMs about attributes. Qwen2-VL-7B [51] presents good visual explainability beyond LLaVA1 5-7B [35] on texts. InternVL2 5-8B [16] focuses on textual content with more red texts and weaker visual activations. 25 [PITH_FULL_IMAGE:f…
Figure 18
Figure 18. Figure 18: TAM supports visual comparison among MLLMs about objects on the COCO Caption [13] dataset. Qwen2-VL-7B [51] presents the best visual results with less correlation (e.g., pizza vs. sandwich in the third row) and shows a stronger activation degree. 26 [PITH_FULL_IMAGE:…
Figure 20
Figure 20. Figure 20: TAM supports explaining attributes of MLLMs at high-quality for the Qwen2-VL-7B [51] about action and colors. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_20.png]
Figure 21
Figure 21. Figure 21: TAM supports explaining attributes of MLLMs at high-quality for the Qwen2-VL-7B [51] about text and shape. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_21.png]
Figure 22
Figure 22. Figure 22: TAM supports explaining attributes of MLLMs at high-quality for both images and videos. The images are processed by the Qwen2-VL-7B [51] from the COCO Caption dataset [13] and we use the Qwen2-VL-2B for videos from the STAR dataset [52]. 29 [PITH_FULL_IMAGE:figures/f…
Figure 23
Figure 23. Figure 23 [PITH_FULL_IMAGE:figures/full_fig_p030_23.png]
Figure 24
Figure 24. Figure 24: TAM supports failure case analysis for deeper understanding with error reason and analysis details using Qwen2-VL-2B [51] on the QK-VQA dataset [37]. 31 [PITH_FULL_IMAGE:figures/full_fig_p031_24.png]
Figure 25
Figure 25. Figure 25: TAM supports failure case analysis for videos with error reason and analysis details using Qwen2-VL-2B [51] on the STAR dataset [52]. 32 [PITH_FULL_IMAGE:figures/full_fig_p032_25.png]
Figure 26
Figure 26. Figure 26: TAM presents good visual explanation result for the VQA dataset with extensive successful examples on the QK-VQA dataset [37] using Qwen2-VL-2B [51]. These cases are dependent on different information, divided into “Object-determined” type and “Textual-determined” typ…
Figure 27
Figure 27. Figure 27: Comparison about video visualization between our TAM and CAM [ [PITH_FULL_IMAGE:figures/full_fig_p034_27.png]
Figure 28
Figure 28. Figure 28: Visual reasoning corner case analyzed using TAM. The analysis reveals that both Qwen2-VL-7B [51] and InternVL2 5-8B [16] incorrectly answered the question due to incomplete recognition of all the arrows. Missing arrows led to erroneous reasoning. Qwen2-VL-7B demonstra…
Figure 29
Figure 29. Figure 29: TAM supports multi-image conversation of MLLM showing wide applicability. It generates high-quality activation maps for four input images in the first case using the Qwen2-VL-2B [51]. In the second case, TAM explains why the model regards the last image as the most in…
Figure 30
Figure 30. Figure 30: TAM supports multi-turn conversation of MLLM for diverse attributes. TAM presents high-quality visual explanation results on Qwen2-VL-2B [51] regarding attributes such as objects, actions, and text in the first round. Then, the user inquires about the fatter dog and t…
Figure 31
Figure 31. Figure 31: TAM enables failure case analysis in multi-turn conversation. Although Qwen2-VL-2B [51] well recognizes objects with good explanation results in the first round chat, it fails to identify motion blur related to speed and mistakenly regards the SUV as the faster car. T…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models

    cs.CV 2025-09 conditional novelty 5.0 of 10

    MSEA+ARC, a multi-scale and ranking-based residualization method, claims consistent F1-IoU gains over TAM for token-level MLLM visual attribution.

Reference graph

Works this paper leans on

65 extracted references · 50 canonical work pages · cited by 1 Pith paper

  1. [1]

    Quantifying attention flow in transformers

    Samira Abnar and Willem Zuidema. Quantifying attention flow in transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 4190–4197, 2020. 2, 5, 6, 7, 14, 15

    work page 2020

  2. [2]

    Attnlrp: Attention- aware layer-wise relevance propagation for transformers

    Reduan Achtibat, Sayed Mohammad Vakilzadeh Hatefi, Maximilian Dreyer, Aakriti Jain, Thomas Wiegand, Sebas- tian Lapuschkin, and Wojciech Samek. Attnlrp: Attention- aware layer-wise relevance propagation for transformers. In International Conference on Machine Learning, pages 135–

    work page

  3. [3]

    Vl-interpret: An interactive visualization tool for interpreting vision-language transformers

    Estelle Aflalo, Meng Du, Shao-Yen Tseng, Yongfei Liu, Chenfei Wu, Nan Duan, and Vasudev Lal. Vl-interpret: An interactive visualization tool for interpreting vision-language transformers. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition , pages 21406– 21415, 2022. 2

    work page 2022

  4. [4]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716–23736,

  5. [5]

    Xai for trans- formers: Better explanations through conservative propa- gation

    Ameen Ali, Thomas Schnake, Oliver Eberle, Gr ´egoire Mon- tavon, Klaus-Robert M ¨uller, and Lior Wolf. Xai for trans- formers: Better explanations through conservative propa- gation. In International Conference on Machine Learning , pages 435–451. PMLR, 2022. 6, 7, 14, 15

    work page 2022

  6. [6]

    Text2live: Text-driven layered image and video editing

    Omer Bar-Tal, Dolev Ofri-Amar, Rafail Fridman, Yoni Kas- ten, and Tali Dekel. Text2live: Text-driven layered image and video editing. In European conference on computer vi- sion, pages 707–723. Springer, 2022. 7

    work page 2022

  7. [7]

    Lvlm-intrepret: An interpretability tool for large vision-language models

    Gabriela Ben Melech Stan, Estelle Aflalo, Raanan Yehezkel Rohekar, Anahita Bhiwandiwalla, Shao-Yen Tseng, Matthew Lyle Olson, Yaniv Gurwicz, Chenfei Wu, Nan Duan, and Vasudev Lal. Lvlm-intrepret: An interpretability tool for large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8182–8187,...

    work page 2024

  8. [8]

    An adaptive median filter for image denoising

    Chin-Chen Chang, Ju-Yuan Hsiao, and Chih-Ping Hsieh. An adaptive median filter for image denoising. In 2008 Second international symposium on intelligent information technol- ogy application, pages 346–350. IEEE, 2008. 3, 5, 6, 16

    work page 2008

Show all 65 references
  1. [9]

    Grad-cam++: General- ized gradient-based visual explanations for deep convolu- tional networks

    Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian. Grad-cam++: General- ized gradient-based visual explanations for deep convolu- tional networks. In 2018 IEEE winter conference on appli- cations of computer vision (WACV) , pages 839–847. IEEE,

    2018

  2. [10]

    Generic attention- model explainability for interpreting bi-modal and encoder- decoder transformers

    Hila Chefer, Shir Gur, and Lior Wolf. Generic attention- model explainability for interpreting bi-modal and encoder- decoder transformers. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 397–406,

  3. [11]

    Transformer inter- pretability beyond attention visualization

    Hila Chefer, Shir Gur, and Lior Wolf. Transformer inter- pretability beyond attention visualization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 782–791, 2021. 2, 5, 15

    2021

  4. [12]

    Less is more: Fewer interpretable region via submodular subset selection

    Ruoyu Chen, Hua Zhang, Siyuan Liang, Jingzhi Li, and Xi- aochun Cao. Less is more: Fewer interpretable region via submodular subset selection. In The Twelfth International Conference on Learning Representations. 3

  5. [13]

    Microsoft coco captions: Data collection and evaluation server

    Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedan- tam, Saurabh Gupta, Piotr Doll ´ar, and C Lawrence Zitnick. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325, 2015. 2, 5, 6, 7, 13, 16, 17, 20, 21, 22, 23, 24, 26, 29, 30

    2015 arXiv

  6. [14]

    Clip-ad: A language-guided staged dual-path model for zero-shot anomaly detection

    Xuhai Chen, Jiangning Zhang, Guanzhong Tian, Haoyang He, Wuhao Zhang, Yabiao Wang, Chengjie Wang, and Yong Liu. Clip-ad: A language-guided staged dual-path model for zero-shot anomaly detection. In International Joint Confer- ence on Artificial Intelligence, pages 17–33. Sprin...

    2024

  7. [15]

    Expanding performance boundaries of open-source multimodal models with model, data, and test- time scaling

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhang- wei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test- time scaling. arXiv preprint arXiv:2412.05271, 2024. 2, 13

    2024 arXiv

  8. [16]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. In Pro- ceedings of the IEEE/CVF conference on computer...

    2024

  9. [17]

    A survey on multimodal large lan- guage models for autonomous driving

    Can Cui, Yunsheng Ma, Xu Cao, Wenqian Ye, Yang Zhou, Kaizhao Liang, Jintai Chen, Juanwu Lu, Zichong Yang, Kuei-Da Liao, et al. A survey on multimodal large lan- guage models for autonomous driving. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vi...

    2024

  10. [18]

    Flashattention: Fast and memory-efficient exact at- tention with io-awareness

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christo- pher R´e. Flashattention: Fast and memory-efficient exact at- tention with io-awareness. Advances in Neural Information Processing Systems, 35:16344–16359, 2022. 6

    2022

  11. [19]

    Vision transformers need registers

    Timoth ´ee Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. In The Twelfth International Conference on Learning Representa- tions. 2, 3, 4

  12. [20]

    Bert: Pre-training of deep bidirectional trans- formers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional trans- formers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the asso- ciation for computational linguistics: human l...

    2019

  13. [21]

    Hia: Towards chinese multimodal llms for comparative high-resolution joint diagnosis

    Xinpeng Ding, Yongqiang Chu, Renjie Pi, Hualiang Wang, and Xiaomeng Li. Hia: Towards chinese multimodal llms for comparative high-resolution joint diagnosis. InInternational Conference on Medical Image Computing and Computer- Assisted Intervention, pages 575–586. Springer, 2024. 7

    2024

  14. [22]

    Holistic autonomous driving un- derstanding by bird’view injected multi-modal large models

    Xinpeng Ding, Jianhua Han, Hang Xu, Xiaodan Liang, Wei Zhang, and Xiaomeng Li. Holistic autonomous driving un- derstanding by bird’view injected multi-modal large models. 9 In 2024 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 13668–13677, 2024. 7

    2024

  15. [23]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. InInternational ...

    2020

  16. [24]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 1, 2

    2016

  17. [25]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perel- man, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Weli- hinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 2

    2024 arXiv

  18. [26]

    Layercam: Exploring hierarchical class activation maps for localization

    Peng-Tao Jiang, Chang-Bin Zhang, Qibin Hou, Ming-Ming Cheng, and Yunchao Wei. Layercam: Exploring hierarchical class activation maps for localization. IEEE Transactions on Image Processing, 30:5875–5888, 2021. 2, 6, 14

    2021

  19. [27]

    Causal inference meets deep learning: A compre- hensive survey

    Licheng Jiao, Yuhan Wang, Xu Liu, Lingling Li, Fang Liu, Wenping Ma, Yuwei Guo, Puhua Chen, Shuyuan Yang, and Biao Hou. Causal inference meets deep learning: A compre- hensive survey. Research, 7:0467, 2024. 2, 3

    2024

  20. [28]

    Unmasking clever hans predictors and as- sessing what machines really learn

    Sebastian Lapuschkin, Stephan W ¨aldchen, Alexander Binder, Gr ´egoire Montavon, Wojciech Samek, and Klaus- Robert M ¨uller. Unmasking clever hans predictors and as- sessing what machines really learn. Nature communications, 10(1):1096, 2019. 2

    2019

  21. [29]

    Llava-docent: Instruction tuning with multimodal large lan- guage model to support art appreciation education

    Unggi Lee, Minji Jeon, Yunseo Lee, Gyuri Byun, Yoorim Son, Jaeyoon Shin, Hongkyu Ko, and Hyeoncheol Kim. Llava-docent: Instruction tuning with multimodal large lan- guage model to support art appreciation education. Comput- ers and Education: Artificial Intelligence , 7:100297...

    2024

  22. [30]

    Manipllm: Embodied multimodal large language model for object-centric robotic manipulation

    Xiaoqi Li, Mingxu Zhang, Yiran Geng, Haoran Geng, Yux- ing Long, Yan Shen, Renrui Zhang, Jiaming Liu, and Hao Dong. Manipllm: Embodied multimodal large language model for object-centric robotic manipulation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pa...

    2024

  23. [31]

    Exploring visual interpretability for con- trastive language-image pre-training

    Yi Li, Hualiang Wang, Yiqun Duan, Hang Xu, and Xi- aomeng Li. Exploring visual interpretability for con- trastive language-image pre-training. arXiv preprint arXiv:2209.07046, 2022. 2

    2022 arXiv

  24. [32]

    A closer look at the explainability of con- trastive language-image pre-training

    Yi Li, Hualiang Wang, Yiqun Duan, Jiheng Zhang, and Xi- aomeng Li. A closer look at the explainability of con- trastive language-image pre-training. Pattern Recognition, page 111409, 2025. 2, 3, 4, 5, 6, 7, 14, 15, 22

    2025

  25. [33]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  26. [34]

    A medical multimodal large language model for future pandemics

    Fenglin Liu, Tingting Zhu, Xian Wu, Bang Yang, Chenyu You, Chenyang Wang, Lei Lu, Zhangdaihong Liu, Yefeng Zheng, Xu Sun, et al. A medical multimodal large language model for future pandemics. NPJ Digital Medicine , 6(1): 226, 2023. 1

    2023

  27. [35]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024. 1, 2, 3, 6, 8, 13, 16, 17, 25

    2024

  28. [36]

    A unified approach to interpreting model predictions

    Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Advances in neural informa- tion processing systems, 30, 2017. 2, 3

    2017

  29. [37]

    Ok-vqa: A visual question answering benchmark requiring external knowledge

    Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowledge. In Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 2, 5, 7, 8, 14, 18, 31, 33

    2019

  30. [38]

    Smooth grad-cam++: An en- hanced inference level visualization technique for deep convolutional neural network models

    Daniel Omeiza, Skyler Speakman, Celia Cintas, and Kom- minist Weldermariam. Smooth grad-cam++: An en- hanced inference level visualization technique for deep convolutional neural network models. arXiv preprint arXiv:1908.01224, 2019. 1, 3

    1908 arXiv

  31. [39]

    Towards vision-language mechanistic interpretabil- ity: A causal tracing tool for blip

    Vedant Palit, Rohan Pandey, Aryaman Arora, and Paul Pu Liang. Towards vision-language mechanistic interpretabil- ity: A causal tracing tool for blip. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 2856–2861, 2023. 3

    2023

  32. [40]

    Causality

    Judea Pearl. Causality. Cambridge university press, 2009. 2, 3

    2009

  33. [41]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  34. [42]

    Glamm: Pixel grounding large multimodal model

    Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, Abdel- rahman Shaker, Salman Khan, Hisham Cholakkal, Rao M Anwer, Eric Xing, Ming-Hsuan Yang, and Fahad S Khan. Glamm: Pixel grounding large multimodal model. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Patter...

    2024

  35. [43]

    ” why should i trust you?” explaining the predictions of any classifier

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. ” why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD interna- tional conference on knowledge discovery and data mining , pages 1135–1144, 2016. 2, 3

    2016

  36. [44]

    Causal interpretation of self-attention in pre-trained trans- formers

    Raanan Y Rohekar, Yaniv Gurwicz, and Shami Nisimov. Causal interpretation of self-attention in pre-trained trans- formers. Advances in Neural Information Processing Sys- tems, 36, 2024. 3

    2024

  37. [45]

    Grad-cam: Visual explanations from deep networks via gradient-based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE in- ternational conference on computer vision , pages 618–626,

  38. [46]

    Training- free object counting with prompts

    Zenglin Shi, Ying Sun, and Mengmi Zhang. Training- free object counting with prompts. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 323–331, 2024. 7 10

    2024

  39. [47]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. 1

    2023 arXiv

  40. [48]

    Understanding how vision-language models rea- son when solving visual math problems

    Joseph Tey. Understanding how vision-language models rea- son when solving visual math problems. 1, 2

  41. [49]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 2, 3

    2017

  42. [50]

    Interpretable bilin- gual multimodal large language model for diverse biomed- ical tasks

    Lehan Wang, Haonan Wang, Honglong Yang, Jiaji Mao, Ze- hong Yang, Jun Shen, and Xiaomeng Li. Interpretable bilin- gual multimodal large language model for diverse biomed- ical tasks. In The Thirteenth International Conference on Learning Representations, 2025. 7

    2025

  43. [51]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024. 1, 2, 5, 6, 7, 8, 13, 14, ...

    2024 arXiv

  44. [52]

    Star: A benchmark for situated reasoning in real-world videos

    Bo Wu, Shoubin Yu, Zhenfang Chen, Joshua B Tenenbaum, and Chuang Gan. Star: A benchmark for situated reasoning in real-world videos. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). 2, 5, 8, 14, 18, 29, 32, 34

  45. [53]

    Efficient streaming language models with attention sinks

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. In The Twelfth International Conference on Learning Representations. 3

  46. [54]

    A survey on causal inference

    Liuyi Yao, Zhixuan Chu, Sheng Li, Yaliang Li, Jing Gao, and Aidong Zhang. A survey on causal inference. ACM Transactions on Knowledge Discovery from Data (TKDD) , 15(5):1–46, 2021. 3

    2021

  47. [55]

    From redundancy to relevance: Enhancing explainability in multimodal large language mod- els

    Xiaofeng Zhang, Yihao Quan, Chen Shen, Xiaosong Yuan, Shaotian Yan, Liang Xie, Wenxiao Wang, Chaochen Gu, Hao Tang, and Jieping Ye. From redundancy to relevance: Enhancing explainability in multimodal large language mod- els. Annual Conference of the Nations of the Americas Ch...

    2025

  48. [56]

    Learning deep features for discrimina- tive localization

    Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discrimina- tive localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2921–2929,

  49. [57]

    with” and the punctuation mark “

    Zijian Zhou, Zheng Zhu, Holger Caesar, and Miaojing Shi. Openpsg: Open-set panoptic scene graph generation via large multimodal models. In European Conference on Com- puter Vision, pages 199–215. Springer, 2024. 2, 5, 6, 7, 14, 16, 17 11 Token Activation Map to Visually Explai...

    2024

  50. [61]

    (2B, 4B), and Qwen2-VL [51] (2B, 7B) across diverse datasets [13, 42, 57] indicates the explainability is increased with more model parameters within a certain range. (d) Increasing the parameters of InternVL2 5 [16] on the OpenPSG dataset [57] improves object precision; howev...

  51. [62]

    living wall

    surpasses LLaV A1 5-7B [35] in text attributes, as il- lustrated in Fig. 17. Furthermore, it demonstrates superior performance in the object cases compared with LLaV A. For instance, Qwen2-VL-7B exhibits stronger activations in the top two rows of Fig. 18 and lacks correlation...

  52. [63]

    living wall

    Specifically, TAM effectively reduces redundant activa- 18 tions and minimizes noise, allowing users to concentrate on target objects and observe the raw video more clearly. Ad- ditionally, we provide case studies in Fig. 25 for video error analysis. R. Corner Case About Reaso...

  53. [64]

    Object-determined

    on the QK-VQA dataset [37]. 31 Type: answer shift by context Details: the second prediction of token “pink” is the right answer. But after generating this context, the final answer is shifted to case from laptop. Type: incomplete answers Details: the given answer is paper / no...

  54. [65]

    Missing arrows led to erroneous reasoning

    incorrectly answered the question due to incomplete recognition of all the arrows. Missing arrows led to erroneous reasoning. Qwen2-VL-7B demonstrated weaker reasoning capabilities compared to InternVL2 5-8B, exhibiting inconsistent context understanding and low text activatio...

  55. [168]

    2, 5, 6, 7, 14, 15

    PMLR, 2024. 2, 5, 6, 7, 14, 15

    2024

  56. [2016]

    2, 6, 16, 17, 18, 20, 24, 30, 34

  57. [2017]

    1, 2, 6, 7, 14, 15, 16, 17, 18, 20, 24, 30, 34

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.