Pith. sign in

REVIEW 2 major objections 4 minor 45 references

Token-level Response-visual Attention Guidance for Multimodal LLMs Knowledge Distillation

T0 review · 2 major / 4 minor · reviewed 2026-07-12 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Response-time visual attention, not prompt attention, is the signal that transfers when compressing multimodal language models, and weighting each token by entropy makes the transfer work.

desk verdict Solid engineering paper: response-to-vision is the useful signal for MLLM attention distillation, and the entropy-weighted token KL (TRAG) delivers real gains on sub-2B students. read the letter →

arxiv 2607.02593 v1 pith:Y7UOTBL6 submitted 2026-07-01 cs.CV cs.CL

Jaehyun Jang , Eunseop Yoon , Hee Suk Yoon , SooHwan Eom , Mark A. Hasegawa-Johnson , Chang D. Yoo This is my paper · ORCID
classification cs.CVcs.CL
keywords knowledgedistillationmultimodallargelanguagemodelscross-modalattentionresponse-to-visionentropy-adaptiveKLvisualgroundingmodelcompression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Compressing multimodal large language models by matching only next-token probabilities leaves the student’s visual grounding weak. Prior attention-distillation methods mainly match how the model looks at the image while reading the prompt. This paper shows that downstream accuracy tracks how closely the student matches the teacher’s attention while generating the answer, and barely tracks prompt-time attention. Attention itself also changes sharply from token to token—diffuse for function words, sharp for visual entities—so a single fixed loss across all tokens is mismatched. The authors therefore introduce TRAG: supervise only response-to-vision attention, average it over intermediate layers, and for each response token blend forward and reverse KL according to the teacher’s attention entropy. The resulting students improve on standard VQA and compositional-reasoning suites and more closely reproduce the teacher’s internal focus during generation.

What carries the argument

Token-level Response-visual Attention Guidance (TRAG): for each response token, form a layer-averaged, vision-normalized attention distribution, then apply a convex combination of forward KL and reverse KL whose weight equals the teacher’s attention entropy (high entropy favors coverage, low entropy favors mode-seeking).

What would settle it

Train identical students with TRAG versus the same response-only objective but without entropy weighting (or with pure forward / pure reverse KL); if the adaptive blend no longer improves both VQA averages and intermediate-layer Response-to-Vision cosine fidelity on held-out data, the central claim fails.

Watch

Extended reading notes

Core claim

Downstream VQA and compositional-reasoning performance of a distilled multimodal student correlates strongly with Response-to-Vision attention similarity to the teacher and negligibly with Prompt-to-Vision similarity; therefore a token-level, entropy-adaptive KL objective on Response-to-Vision attention (TRAG) yields larger gains than prior prompt-focused or uniform attention distillation.

Load-bearing premise

That teacher-forced attention maps taken from intermediate layers and averaged once per ground-truth response token remain a good enough teacher for the student when it later generates freely without teacher forcing.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper argues that standard logit-only KD is insufficient for MLLMs because it does not transfer visual grounding, and that prior attention-distillation methods are suboptimal because they focus on Prompt-to-Vision maps and apply uniform objectives. Through correlation analysis on held-out data (Fig. 3) and layer/token visualizations (Fig. 4), the authors claim that Response-to-Vision attention similarity to the teacher strongly predicts downstream VQA and compositional-reasoning performance while Prompt-to-Vision similarity does not, and that attention patterns vary sharply across response tokens. They therefore introduce TRAG, which (i) restricts cross-modal supervision to Response-to-Vision maps aggregated over intermediate layers and (ii) applies a per-token mixture of forward and reverse KL weighted by the teacher’s attention entropy (Eqs. 11–14). Integrated into a three-stage DPT–SFT–DFT pipeline, TRAG is shown to outperform LLaVA-KD, CompoDistill and other baselines on eight VQA and three CR benchmarks across Qwen1.5/2.5 and MobileLLaMA backbones, while also raising free-generation AR→V fidelity.

Significance. If the empirical findings hold, TRAG supplies a concrete, easy-to-implement improvement to the dominant MLLM distillation pipeline and supplies evidence that generative-phase (rather than prompt-phase) visual grounding is the more transferable signal. The multi-backbone results, controlled ablations on query type and matching objective (Tables 3–4), attention-fidelity measurements under free generation, and public code constitute reproducible contributions that are of practical value for deploying sub-2B multimodal models. The entropy-adaptive KL construction is a modest but well-motivated technical device that could be reused beyond the present setting.

major comments (2)
  1. [§4.1 / Fig. 3] Section 4.1 and Fig. 3 present the central motivational claim that AR→V similarity (but not AP→V) correlates with downstream performance. The plots contain only four points per panel drawn from models that already differ substantially in overall capacity and training recipe (SFT 0.5B, LLaVA-KD, CompoDistill, SFT 1.8B). Because stronger students tend to produce higher-quality free-running generations and therefore higher teacher-forced AR→V maps, the observed correlation is vulnerable to a general-quality confound. A controlled experiment that varies only the attention target while holding the rest of the distillation objective fixed would make the causal claim load-bearing rather than merely suggestive.
  2. [§5 / §6.3 / Limitations H] The TRAG objective (Eq. 14) and all training stages operate under teacher forcing on ground-truth response tokens, yet the fidelity evaluation in §6.3 and the Limitations (§H) acknowledge that inference is free-running. While post-hoc cosine similarity reaches ~0.72 at intermediate layers, the paper never optimizes or ablate an on-policy variant in which the teacher’s attention is queried on the student’s own generated tokens. Given that the authors themselves flag this train–inference discrepancy as a primary limitation, an on-policy experiment (or a clear demonstration that the gap is negligible) is needed to underwrite the claim that the distilled maps remain faithful at deployment.
minor comments (4)
  1. [Table 1] Table 1 mixes models trained with different sample counts (1.2 M vs 5 M for LLaVA-MoD). While the authors note the discrepancy, a short column or footnote quantifying the sample-matched gap would make the comparison cleaner.
  2. [Eq. 12 / §5.2] The EMA momentum β = 0.995 and the precise relative-depth interval [0.3, 0.6] are free parameters; a one-sentence sensitivity statement (or reference to the existing depth ablation in App. F) would help readers judge robustness.
  3. [Fig. 1] In Fig. 1 the notation for the adaptive weight λi is introduced before the entropy definition; a forward reference to Eqs. 11–12 would improve readability.
  4. [§3.2] Appendix C’s decomposition of CompoDistill is useful; a short sentence in the main text pointing to it would help readers who skip the supplement.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TRAG is an empirical distillation design evaluated on held-out VQA/CR benchmarks; no claim reduces by construction to its own inputs.

full rationale

The paper’s load-bearing chain is observational then constructive, not definitional. Section 4.1 measures teacher–student cosine similarity of Prompt-to-Vision vs Response-to-Vision attention on the external set VIGC-InstData and reports correlation with downstream VQA/CR scores across a small set of differently trained students; that correlation motivates shifting supervision to Response-to-Vision (Eq. 7) and token-wise entropy-adaptive forward/reverse KL (Eqs. 11–14). Entropy bounds H_min/H_max are online EMAs during training, not fitted to test metrics. Final claims are accuracy and fidelity numbers on standard held-out suites (Tables 1–2, Fig. 5) and ablations (Tables 3–4), none of which equal the training loss by construction. Self-citations (LLaVA-KD uni-modal terms, CompoDistill as a baseline, co-author PDCR/on-policy notes in App. G/H) supply pipeline components or future-work pointers; they do not force the reported gains or uniqueness of AR→V. Confounding of correlation by overall student quality is a causal-identification concern, not circularity under the stated patterns. Derivation is therefore self-contained against external benchmarks.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The work is empirical method design. It inherits standard Transformer attention, KL divergence properties, and the LLaVA three-stage distillation pipeline. Free parameters are ordinary training hyper-parameters and the hand-chosen intermediate depth band; no new physical constants or fitted scaling laws appear. Invented entities are purely algorithmic constructs (TRAG loss, entropy weight λi) with no claim of independent ontological status.

free parameters (4)
  • relative depth band for layer aggregation = [0.3, 0.6]
    Hand-chosen interval [0.3,0.6] (ablated to [0.4,0.7]); performance depends on supervising intermediate layers.
  • EMA momentum β for Hmin/Hmax = 0.995
    Online min–max normalization of teacher attention entropy; set to 0.995 without further search reported.
  • uni-modal loss weights λα, λβ = 1.0 / 0.5
    Inherited from LLaVA-KD (1.0 and 0.5) and kept fixed.
  • learning rates and batch sizes per stage = 1e-3 / 2e-5
    Standard AdamW schedule (1e-3 DPT, 2e-5 SFT/DFT; global BS 256/128); ordinary training knobs.
assumptions (4)
  • standard math Causal self-attention in Transformer decoders yields well-defined cross-modal submatrices from textual queries to visual keys.
    Used throughout §3.1 to define AP→V and AR→V.
  • domain assumption Forward KL is mean-seeking (coverage) and reverse KL is mode-seeking (precision).
    Invoked in §5.2 to justify the entropy-gated mixture; standard information-geometry folklore, not re-proved.
  • domain assumption Intermediate decoder layers (relative depth ~0.3–0.6) concentrate the most informative cross-modal interactions.
    Cited from prior VLM interpretability work [7–9,19,29] and used to select MS, MT.
  • ad hoc to paper Teacher-forced attention maps under ground-truth response tokens are a useful proxy for the grounding behavior needed at free-running inference.
    Implicit in the entire TRAG objective; Limitations §H acknowledges residual train–inference mismatch.
invented entities (2)
  • TRAG (Token-level Response-visual Attention Guidance) objective
    purpose: Token-wise entropy-weighted combination of forward and reverse KL on layer-aggregated Response-to-Vision attention.
    Algorithmic construct defined by Eqs. 8–14; no claim of existence outside the loss.
  • Entropy-driven adaptive weight λi
    purpose: Maps teacher attention entropy to the mixture coefficient between forward and reverse KL for each response token.
    Defined by min–max normalization of H(ãTi); purely a scheduling device.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Token-level Response-visual Attention Guidance for Multimodal LLMs Knowledge Distillation." pith.science (2026). https://pith.science/paper/Y7UOTBL6

@misc{pith2026260702593,
  author       = {Pith},
  title        = {Pith review of: Token-level Response-visual Attention Guidance for Multimodal LLMs Knowledge Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y7UOTBL6}},
  note         = {Machine review of arXiv:2607.02593}
}
read the original abstract

While knowledge distillation (KD) is widely adopted for training lightweight models by leveraging supervision from larger teacher models, relying solely on output token distributions has proven insufficient for compressing Multimodal Large Language Models (MLLMs). Since output tokens are a byproduct of the model attending to visual inputs, prior works have explored explicitly distilling attention to provide a direct supervisory signal. While promising, the precise utility of which attention signals to distill remains under-explored. In this work, we challenge the conventional reliance on prompt-to-vision attention by revealing that downstream performance correlates strongly with response-to-vision attention similarity to the teacher, but negligibly with that of prompt-conditioned attention. Furthermore, we observe that attention distributions exhibit significant variance across individual tokens, indicating that a uniform distillation objective is suboptimal. To this end, we introduce Token-level Response-visual Attention Guidance (TRAG), a distillation objective that 1) shifts the focus to response-to-vision signals and 2) employs token-specific objectives by adaptively weighting the Kullback-Leibler divergence based on attention entropy, effectively guiding the student to mirror the teacher's precise visual focus. Extensive experimental results on multiple benchmarks demonstrate that TRAG significantly outperforms prior distillation baselines.

Figures

Figures reproduced from arXiv: 2607.02593 by the authors.

Figure 1
Figure 1. Overview of Token-level Response-visual Attention Guidance [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Comparison of attention extraction according to query source. Standard MLLMs encode an input image into visual tokens v = {v1, . . . , vN } via a vision encoder and an MLP-based projection mod￾ule [24]. The model then processes the con￾catenated sequence c = [v; x; y], where x = {x1, . . . , xLP } denotes the prompt token se￾quence and y = {y1, . . . , yLR } denotes the au￾toregressive response token sequence. Letti… view at source ↗
Figure 3
Figure 3. Correlation analysis between attention similarity and downstream [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Analysis of attention granularity across layers and tokens. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Layer-wise Response-to-Vision atten￾tion fidelity. We evaluate attention fidelity on VIGC-InstData [35] using layer￾wise teacher–student (AR→V ) co￾sine similarity during autoregres￾sive generation (without teacher forcing), following the similarity definition in Secti…
Figure 6
Figure 6. Figure 6: Illustration of the Compositional Reasoning Evaluation setup. [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Token-level attention adaptation for action versus object tokens. [PITH_FULL_IMAGE:figures/full_fig_p026_7.png]
Figure 8
Figure 8. Figure 8: Attention dynamics for spatial relationships and entities. [PITH_FULL_IMAGE:figures/full_fig_p027_8.png]
Figure 9
Figure 9. Figure 9: Variance in spatial focus between context-setting and action-specific [PITH_FULL_IMAGE:figures/full_fig_p027_9.png]
Figure 10
Figure 10. Figure 10: Qualitative comparison on scene description and salient feature [PITH_FULL_IMAGE:figures/full_fig_p032_10.png]
Figure 11
Figure 11. Figure 11: Qualitative analysis of counting and precise object identification. [PITH_FULL_IMAGE:figures/full_fig_p033_11.png]
Figure 12
Figure 12. Figure 12: Demonstration of capturing fine-grained object attributes. [PITH_FULL_IMAGE:figures/full_fig_p034_12.png]
Figure 13
Figure 13. Figure 13: Qualitative evidence for replicating descriptive response patterns. [PITH_FULL_IMAGE:figures/full_fig_p035_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 4 linked inside Pith

  1. [1]

    In: The Twelfth International Conference on Learning Representations (2024) 2

    Agarwal, R., Vieillard, N., Zhou, Y., Stanczyk, P., Garea, S.R., Geist, M., Bachem, O.: On-policy distillation of language models: Learning from self-generated mis- takes. In: The Twelfth International Conference on Learning Representations (2024) 2

    2024

  2. [2]

    arXiv preprint arXiv:2212.12965 (2022) 10

    Amara, I., Sepahvand, N., Meyer, B.H., Gross, W.J., Clark, J.J.: Bd-kd: Balancing the divergences for online knowledge distillation. arXiv preprint arXiv:2212.12965 (2022) 10

    arXiv 2022

  3. [3]

    Jang et al

    Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Ge, W., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., Li, M., Li, K., Lin, Z., Lin, J., Liu, X., Liu, J., Liu, C., Liu, Y., Liu, D., Liu, S., Lu, D., Luo, R., Lv, C., Men, R., Meng, L., Ren, X., Ren, X., Song, S., Sun, Y., Tang, ...

    2025

  4. [4]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Cai, Y., Zhang, J., He, H., He, X., Tong, A., Gan, Z., Wang, C., Xue, Z., Liu, Y., Bai, X.: Llava-kd: A framework of distilling multimodal large language models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 239–249 (2025) 3, 5, 10, 24, 30

    2025

  5. [5]

    Cao, J., Zhang, Y., Huang, T., Lu, M., Zhang, Q., An, R., Ma, N., Zhang, S.: Move- kd:Knowledgedistillationforvlmswithmixtureofvisualencoders.In:Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR). pp. 19846– 19856 (June 2025) 3

    2025

  6. [6]

    arXiv preprint arXiv:2602.09483 (2026) 3

    Chen, L., Zhao, X., Ding, K., Feng, W., Miao, C., Wang, Z., Guo, W., Wang, Y., Zheng, K., Zhang, B., Li, Z., Xiang, S.: Beyond next-token alignment: Dis- tilling multimodal large language models via token interactions. arXiv preprint arXiv:2602.09483 (2026) 3

    arXiv 2026

  7. [7]

    arXiv preprint arXiv:2503.01773 (2025) 8, 13

    Chen, S., Zhu, T., Zhou, R., Zhang, J., Gao, S., Niebles, J.C., Geva, M., He, J., Wu, J., Li, M.: Why is spatial reasoning hard for vlms? an attention mechanism perspective on focus areas. arXiv preprint arXiv:2503.01773 (2025) 8, 13

    arXiv 2025

  8. [8]

    Scientific Reports (2026) 8

    Chen, Y., Wang, P., Qin, G., Wu, W., Chen, M., Hao, Y.: Attention re-alignment in multimodal large language models via intermediate-layer guidance. Scientific Reports (2026) 8

    2026

Show all 45 references
  1. [9]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Fan,Y.,Tong,J.,Zhao,A.,Shen,X.:Whatdovisualtokensreallyencode?uncover- ing sparsity and redundancy in multimodal large language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11987–11997 (2026) 8

    2026

  2. [10]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Feng, Q., Li, W., Lin, T., Chen, X.: Align-kd: Distilling cross-modal alignment knowledge for mobile vision-language large model enhancement. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 4178–4188 (2025) 2, 3, 6

    2025

  3. [11]

    In: The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track (2025) 11, 22

    Fu, C., Chen, P., Shen, Y., Qin, Y., Zhang, M., Lin, X., Yang, J., Zheng, X., Li, K., Sun, X., et al.: Mme: A comprehensive evaluation benchmark for multimodal large language models. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and B...

    2025

  4. [12]

    In: The Twelfth International Conference on Learning Represen- tations (2024) 2

    Gu, Y., Dong, L., Wei, F., Huang, M.: MiniLLM: Knowledge distillation of large language models. In: The Twelfth International Conference on Learning Represen- tations (2024) 2

    2024

  5. [13]

    Image and Vision Computing146, 105020 (2024) 3

    Guo, Z., Zhang, P., Liang, P.: Sakd: Sparse attention knowledge distillation. Image and Vision Computing146, 105020 (2024) 3

    2024

  6. [14]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Guo, Z., Yan, H., Li, H., Lin, X.: Class attention transfer based knowledge distilla- tion. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 11868–11877 (2023) 3

    2023

  7. [15]

    arXiv preprint arXiv:1606.08415 (2016) 11

    Hendrycks, D.: Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415 (2016) 11

    2016 arXiv

  8. [16]

    Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network (2015) 1, 5

    2015

  9. [17]

    Advances in neural in- formation processing systems36, 31096–31116 (2023) 11, 21, 22

    Hsieh, C.Y., Zhang, J., Ma, Z., Kembhavi, A., Krishna, R.: Sugarcrepe: Fixing hackable benchmarks for vision-language compositionality. Advances in neural in- formation processing systems36, 31096–31116 (2023) 11, 21, 22

    2023

  10. [18]

    In: Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition

    Hudson, D.A., Manning, C.D.: Gqa: A new dataset for real-world visual reasoning and compositional question answering. In: Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition. pp. 6700–6709 (2019) 11, 22

    2019

  11. [19]

    In: The Fourteenth International Con- ference on Learning Representations (2026) 2, 3, 5, 6, 8, 10, 13, 22, 24 Token-level Response-visual Attention Guidance for MLLMs KD 17

    Kim, J., Kim, K., Seo, S., Park, C.: Compodistill: Attention distillation for com- positional reasoning in multimodal LLMs. In: The Fourteenth International Con- ference on Learning Representations (2026) 2, 3, 5, 6, 8, 10, 13, 22, 24 Token-level Response-visual Attention Guid...

    2026

  12. [20]

    In: Forty-first International Conference on Machine Learning (2024) 2

    Ko, J., Kim, S., Chen, T., Yun, S.Y.: DistiLLM: Towards streamlined distillation for large language models. In: Forty-first International Conference on Machine Learning (2024) 2

    2024

  13. [21]

    In: The 2023 Conference on Empirical Methods in Natural Language Processing 11, 22

    Li, Y., Du, Y., Zhou, K., Wang, J., Zhao, X., Wen, J.R.: Evaluating object hal- lucination in large vision-language models. In: The 2023 Conference on Empirical Methods in Natural Language Processing 11, 22

    2023

  14. [22]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Lin, J., Yin, H., Ping, W., Molchanov, P., Shoeybi, M., Han, S.: Vila: On pre- training for visual language models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 26689–26699 (2024) 1

    2024

  15. [23]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Liu, H., Li, C., Li, Y., Lee, Y.J.: Improved baselines with visual instruction tun- ing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 26296–26306 (2024) 11, 21

    2024

  16. [24]

    Advances in neural information processing systems36, 34892–34916 (2023) 1, 4

    Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. Advances in neural information processing systems36, 34892–34916 (2023) 1, 4

    2023

  17. [25]

    Liu, Y., Duan, H., Zhang, Y., Li, B., Zhang, S., Zhao, W., Yuan, Y., Wang, J., He, C., Liu, Z., et al.: Mmbench: Is your multi-modal model an all-around player? In: European conference on computer vision. pp. 216–233. Springer (2024) 11, 22

    2024

  18. [26]

    arXiv preprint arXiv:2403.05525 (2024) 1

    Lu, H., Liu, W., Zhang, B., Wang, B., Dong, K., Liu, B., Sun, J., Ren, T., Li, Z., Yang, H., et al.: Deepseek-vl: towards real-world vision-language understanding. arXiv preprint arXiv:2403.05525 (2024) 1

    2024 arXiv

  19. [27]

    Advances in Neural Information Processing Systems 35, 2507–2521 (2022) 11, 22

    Lu, P., Mishra, S., Xia, T., Qiu, L., Chang, K.W., Zhu, S.C., Tafjord, O., Clark, P., Kalyan, A.: Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems 35, 2507–2521 (2022) 11, 22

    2022

  20. [28]

    Advances in Neural Information Processing Systems37, 101880–101904 (2024) 11, 21, 22

    Miranda, I., Salaberria, A., Agirre, E., Azkune, G.: Bivlc: Extending vision- language compositionality evaluation with text-to-image retrieval. Advances in Neural Information Processing Systems37, 101880–101904 (2024) 11, 21, 22

    2024

  21. [29]

    In: International Confer- ence on Learning Representations

    Neo, C., Ong, L., Torr, P., Geva, M., Krueger, D., Barez, F.: Towards interpreting visual information processing in vision-language models. In: International Confer- ence on Learning Representations. vol. 2025, pp. 57172–57189 (2025) 8

    2025

  22. [30]

    In: Pro- ceedings of the 61st Annual Meeting of the Association for Computational Linguis- tics (Volume 1: Long Papers)

    Pandey, R., Shao, R., Liang, P.P., Salakhutdinov, R., Morency, L.P.: Cross-modal attention congruence regularization for vision-language relation alignment. In: Pro- ceedings of the 61st Annual Meeting of the Association for Computational Linguis- tics (Volume 1: Long Papers)....

    2023

  23. [31]

    In: The Thirteenth International Conference on Learning Representations (2025) 3, 22

    Shu, F., Liao, Y., Zhang, L., Zhuo, L., Xu, C., Zhang, G., Shi, H., Chan, L., Yu, Z., He, W., et al.: Llava-mod: Making llava tiny via moe-knowledge distillation. In: The Thirteenth International Conference on Learning Representations (2025) 3, 22

    2025

  24. [32]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Singh, A., Natarajan, V., Shah, M., Jiang, Y., Chen, X., Batra, D., Parikh, D., Rohrbach, M.: Towards vqa models that can read. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8317–8326 (2019) 11, 22

    2019

  25. [33]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition

    Thrush, T., Jiang, R., Bartolo, M., Singh, A., Williams, A., Kiela, D., Ross, C.: Winoground: Probing vision and language models for visio-linguistic composition- ality. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition. pp. 5238–5248 (202...

    2022

  26. [34]

    Advances in neural information pro- cessing systems30(2017) 4 18 J

    Vaswani,A.,Shazeer,N.,Parmar,N.,Uszkoreit,J.,Jones,L.,Gomez,A.N.,Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information pro- cessing systems30(2017) 4 18 J. Jang et al

    2017

  27. [35]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Wang, B., Wu, F., Han, X., Peng, J., Zhong, H., Zhang, P., Dong, X., Li, W., Li, W., Wang, J., et al.: Vigc: Visual instruction generation and correction. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 5309– 5317 (2024) 6, 13

    2024

  28. [36]

    Advances in Neural Information Processing Systems37, 114553–114573 (2024) 1

    Wang, C., Wang, Z., Xu, X., Tang, Y., Zhou, J., Lu, J.: Q-vlm: Post-training quantization for large vision-language models. Advances in Neural Information Processing Systems37, 114553–114573 (2024) 1

    2024

  29. [37]

    In: Pro- ceedings of the 31st International Conference on Computational Linguistics

    Wu, T., Tao, C., Wang, J., Yang, R., Zhao, Z., Wong, N.: Rethinking kullback- leibler divergence in knowledge distillation for large language models. In: Pro- ceedings of the 31st International Conference on Computational Linguistics. pp. 5737–5755 (2025) 10

    2025

  30. [38]

    arXiv preprint arXiv:2407.19409 (2024) 2, 3

    Xu, S., Li, X., Yuan, H., Qi, L., Tong, Y., Yang, M.H.: Llavadi: What matters for multimodal large language models distillation. arXiv preprint arXiv:2407.19409 (2024) 2, 3

    2024 arXiv

  31. [39]

    Scientific Reports13(1), 18369 (2023) 4

    Yang, G., Yu, S., Sheng, Y., Yang, H.: Attention and feature transfer based knowl- edge distillation. Scientific Reports13(1), 18369 (2023) 4

    2023

  32. [40]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Yoon, H.S., Yoon, E., Hong, J.W., Eom, S., Koo, G., Hasegawa-Johnson, M., Dai, Q., Luo, C., Yoo, C.D.: Pdcr: Perception-decomposed confidence reward for vision- language reasoning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 18881...

    2026

  33. [41]

    In: Forty-third International Conference on Machine Learning (2026) 2, 31

    Yoon, H.S., Yoon, E., Jang, J., Eom, S., Hong, J.W., Hasegawa-Johnson, M.A., Dai, Q., Luo, C., Yoo, C.D.: Decomposed on-policy distillation for vision-language reasoning: Steering gradients for visual grounding. In: Forty-third International Conference on Machine Learning (2026) 2, 31

    2026

  34. [42]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Yue, X., Ni, Y., Zhang, K., Zheng, T., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., et al.: Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Patt...

    2024

  35. [43]

    In: Interna- tional Conference on Learning Representations (2017) 3

    Zagoruyko, S., Komodakis, N.: Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. In: Interna- tional Conference on Learning Representations (2017) 3

    2017

  36. [44]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Zhai, X., Mustafa, B., Kolesnikov, A., Beyer, L.: Sigmoid loss for language im- age pre-training. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 11975–11986 (2023) 11

    2023

  37. [45]

    an airport

    Zhang, Q., Cheng, A., Lu, M., Zhang, R., Zhuo, Z., Cao, J., Guo, S., She, Q., Zhang, S.: Beyond text-visual attention: Exploiting visual cues for effective token pruning in vlms. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 20857–20867 (2025...

    2025

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.