Pith. sign in

REVIEW 5 major objections 5 minor 39 references

A 1.2B-parameter vision-language model matches 7B-13B rivals on visual QA.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 20:14 UTC pith:365B6D4E

load-bearing objection A plausible lightweight VLM whose headline GQA number rests on a split-disclosure the authors never make; fix that and the parameter count and it's a legitimate contribution. the 5 major comments →

arxiv 2509.08715 v1 pith:365B6D4E submitted 2025-09-10 cs.CV

BcQLM: Efficient Vision-Language Understanding with Distilled Q-Gated Cross-Modal Fusion

Sike Xiang , Shuang Chen , Amir Atapour-Abarghouei This is my paper
classification cs.CV
keywords lightweight multimodal large language modelvisual question answeringknowledge distillationcontrastive learningcross-modal fusionBreezeCLIPQ-Gated Cross-Attentionefficient inference
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to show that a small vision-language model can reach the accuracy of much larger multimodal LLMs on visual question answering. It builds BcQLM around BreezeCLIP, a 31-million-parameter encoder distilled from a CLIP teacher with contrastive and distillation losses, and feeds its features through a question-gated cross-modal fusion module into a 1B LLaMA decoder. At 1.2B total parameters, BcQLM reports 62.4% on GQA, 78.7% on VQAv2, and 56.1% on VizWiz at 336x336 input, placing it near 7B-13B models on those benchmarks while running roughly twice as fast as 3-4B rivals with about 30% less memory. If the results hold, they would make deployable on-device multimodal QA practical on modest hardware without giving up benchmark accuracy.

Core claim

The central claim is that a 1.2B-parameter vision-language model, built from a distilled 31M-parameter dual encoder and a question-conditioned gated fusion module, performs comparably to mainstream 7B-13B MLLMs on three VQA benchmarks. BreezeCLIP is trained by jointly minimizing an InfoNCE contrastive loss and an MSE distillation loss against a frozen CLIP ViT-B/32 teacher on image-text pairs derived from GQA scene graphs; the resulting student reaches over 80% mean cosine similarity on positive pairs, above the teacher's ~71%. The Q-Gated Cross-Modal Attention module uses the pooled question embedding to compute a per-patch gate, modulating attention-enhanced visual features through a gated

What carries the argument

BreezeCLIP: a lightweight dual encoder using inverted-bottleneck blocks for images (MobileNetV2-style) and bottleneck Transformer layers for text (MobileBERT-style), with factorized embeddings, totaling 31M parameters versus ~151M for standard CLIP. Q-GCAM (Q-Gated Cross-Attention Module): visual patch tokens attend to text tokens; the pooled text vector is projected to a gate gamma in [0,1] per spatial location, and the attention-enhanced visual features are fused with the original features via F_mod = F_I + gamma * F_hat_I before a feed-forward and normalization layer; the output is mapped by an adapter to pseudo text tokens for the LLaMA decoder. The joint training objective L_total = lam

Load-bearing premise

BreezeCLIP is pretrained on image-text pairs built from GQA scene graphs, and GQA is also one of the three evaluation benchmarks; the claim of GQA parity assumes the pretraining images and the evaluated GQA split share no images.

What would settle it

Inspect the GQA split used to build BreezeCLIP's pretraining pairs and compare it against the GQA split reported in Table 1; if any pretraining image overlaps the evaluation set, the 62.4% GQA score is contaminated. A definitive test: retrain BreezeCLIP with image-text pairs drawn from COCO captions only and see whether GQA accuracy stays near 62%.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • A 1.2B model (31M visual-text encoder + 1B decoder) can reach 62.4% GQA, 78.7% VQAv2, and 56.1% VizWiz at 336x336, ranking among published 7B-13B models on these three benchmarks.
  • Inference on an RTX 4070 Ti costs about 127 ms per sample, 4989.5 MiB peak memory, and 1.57e11 FLOPs — roughly twice the speed and 30% lower memory than Qwen2.5-VL-3B and Gemma3-4B.
  • Distilling CLIP's alignment into a 31M student can yield higher positive-pair cosine similarity (80%+) than the teacher (71%) on the distillation set, indicating alignment need not be tied to large backbones.
  • The Q-GCAM gate is the main accuracy driver: removing it (BreezeCLIP+LLaMA) drops GQA from 60.8% to 59.2%, and alternate fusion variants (Token Balance, Visual Query) perform worse.
  • Because the decoder stays mostly frozen (5% unfreezing suffices for semantically correct answers), the fusion module rather than decoder fine-tuning is what enables cross-modal grounding.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to pretrain BreezeCLIP on image-text pairs from a disjoint corpus (for example COCO captions) and re-measure GQA, which would separate genuine cross-modal transfer from dataset familiarity.
  • The >80% cosine similarity on positive pairs could reflect overfitting to GQA-specific pairings; zero-shot image-text retrieval on held-out sets such as COCO or Flickr30K would test whether the distilled alignment generalizes.
  • The question-gated residual fusion is task-agnostic in form and could be transferred to referring expression comprehension, visual grounding, or video-text QA, where the question/query plays the same gating role.
  • The efficiency comparison (2x speed, 30% lower memory) is measured at a single batch of 20 samples on one GPU; throughput at different batch sizes, sequence lengths, and hardware would show whether the advantage holds for production serving.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes BcQLM, a lightweight visual-question-answering system composed of three components: BreezeCLIP, a 31M-parameter dual encoder trained via contrastive learning and CLIP distillation on image-text pairs derived from GQA scene graphs; a Q-Gated Cross-Attention Module (Q-GCAM) that fuses visual and textual features; and a LLaMA-3.2-1B decoder that generates answers. The authors report results on GQA, VQAv2, and VizWiz, with accuracy at 224x224 of 60.8/71.0/49.5 and at 336x336 of 62.4/78.7/56.1, and claim that these numbers are comparable to standard-size MLLMs while using only about 1.2B total parameters. The paper also includes an ablation study of Q-GCAM and an efficiency comparison showing lower memory and latency than Qwen2.5-VL-3B and Gemma3-4B.

Significance. If the claims hold, the paper would be a useful contribution to efficient multimodal modeling: a 1B-scale decoder with a 31M distilled encoder reaching accuracy roughly on par with 7B-13B MLLMs on three established VQA benchmarks would be a practical result for resource-constrained deployment. The paper has strengths: it releases code, describes a modular architecture, reports an ablation, and provides concrete efficiency measurements on a common GPU. The significance is conditional, however, because the headline GQA result is tied to the fact that the same dataset (GQA) is used both to construct BreezeCLIP training pairs and as an evaluation benchmark, and the split relationship is never disclosed. The parameter-count and ranking claims in Table 1 also need correction before the central efficiency/comparability argument can be accepted.

major comments (5)
  1. [§3.1, §4.1, §4.3, Table 1] GQA is used both as the source of BreezeCLIP pretraining pairs and as an evaluation benchmark, but the manuscript never states which splits are used for either purpose. §3.1 says image-text pairs are constructed by leveraging scene graphs in the GQA dataset; §4.1 evaluates on GQA; §4.3 mentions computing InfoNCE loss on 'the validation set' without identifying that set. If any GQA images used in evaluation appeared in BreezeCLIP pretraining or decoder training, the 62.4% GQA score in Table 1 is inflated and undermines the comparability claim. Please state explicitly that only the GQA train split was used for all training, that the evaluation images were disjoint, and report the exact evaluation split (e.g., GQA test-dev or val).
  2. [Table 1, §1, §4.4] The '1.2B total parameters' claim is inconsistent with the components. LLaMA-3.2-1B has approximately 1.23B parameters, BreezeCLIP is stated as 31M, and the projection head/adapter adds more, so the total is about 1.26B, not 1.2B. Since parameter efficiency is a core contribution, please provide a precise parameter breakdown for the full model and use consistent counting for baselines. The statement that BreezeCLIP is '10% of SoTA' also depends on which SoTA model is meant; Table 1 includes models up to 80B, so the comparison base should be defined.
  3. [Table 1, §4.4] The ranking claims in §4.4 are not supported by the table. At 336x336, BcQLM scores 62.4 on GQA, below LLaVA-1.5-13B (63.3) and VILA-13B (63.3), so 'second best' is inaccurate; on VQAv2, BcQLM at 78.7 is below 80.8, 80.0, 79.9, and 78.8 in the same table, so 'third best' requires clarification. Please either correct the ranking statements, restrict the comparison set explicitly, or show that the baseline numbers are not directly comparable due to different evaluation protocols.
  4. [§4.3, §4.4] The evaluation protocol is underspecified. The paper says 'VQA accuracy' is the sole metric but does not define how GQA's structured answers, VQAv2's ten-answer agreement, or VizWiz's answer-merging are scored; it also does not state whether GQA/VQAv2 numbers are on val or test-dev. Because the paper compares with published numbers from other models, please specify exact evaluation splits and metric implementations for every benchmark. Without this, the reader cannot determine whether the comparisons in Table 1 are apples-to-apples.
  5. [§4.4, Figure 2] The comparison between BreezeCLIP and teacher CLIP cosine similarity is misleading as presented. BreezeCLIP is trained on GQA-derived pairs, while the CLIP teacher is not; observing higher positive-pair similarity for the student therefore reflects training data exposure as much as encoder quality. If the purpose is to demonstrate distillation effectiveness, report the same similarity metrics on a held-out set that was not used for BreezeCLIP pretraining. This also connects to the split-disclosure issue in §3.1.
minor comments (5)
  1. [§3.1, Eq. (1)] The cross-entropy notation CE(·, y) in Eq. (1) is never defined; specify that it is the symmetric InfoNCE loss and clarify how labels y are constructed for each directional term.
  2. [Table 2, §4.5] The ablation uses 'the validation set' without naming the dataset or split. State clearly which benchmark and split Table 2 refers to, and provide at least three random seeds or confidence intervals for the accuracy differences; the reported gaps (58.8 vs 59.2, 58.7 vs 55.7) may be within noise.
  3. [Table 1] The table header writes 'VisWiz' while the text and references use 'VizWiz'; correct the typo. Also, the baseline results are reported as 'based on official releases' but no exact checkpoint or evaluation configuration is given for each; add a footnote with the specific split and metric used.
  4. [§4.6] Efficiency is measured on 20 test samples, which is a very small sample for latency and memory measurements. Report the standard deviation over multiple runs or increase the sample size, and specify input sequence length, batch size, and hardware details beyond the GPU model.
  5. [§3.2, Eq. (5)] The notation FI, FT, and Fmod appears in Eq. (5)-(10) before being fully introduced; a sentence linking these symbols to the preceding definitions would improve readability.

Circularity Check

0 steps flagged

No demonstrated circularity; GQA pretraining/evaluation overlap is an undisclosed data-hygiene risk, not a derivation-equivalent step.

full rationale

The claimed derivation chain is not circular in the formal sense. BreezeCLIP is trained with explicit objectives in Eqs. (1)-(3): an InfoNCE contrastive term and an MSE distillation term against a frozen CLIP teacher; the Q-GCAM fusion (Eqs. (4)-(11)) and the LLaMA-3.2-1B decoder loss (Eq. (13)) are standard, externally checkable components. The headline comparisons in Table 1 are against published baseline numbers taken from official releases, and the model's VQAv2 (78.7) and VizWiz (56.1) results are on datasets not used for BreezeCLIP pretraining, so the central efficiency/accuracy claim has independent support. The one legitimate concern is that Section 3.1 constructs BreezeCLIP's image-text pretraining pairs from GQA scene graphs while Section 4.4 reports GQA as the best result; the paper never states which GQA split is used for pretraining versus evaluation. This is a potential data-contamination / split-disclosure problem, but it is not a demonstrated equivalence of input and output: the GQA accuracy is not by construction equal to the training objective, and no equation or self-citation forces the reported number. Section 4.4's claim that BreezeCLIP 'achieves stronger alignment' than the teacher is computed on the model's own training pairs and is therefore in-sample evidence, but it is not the load-bearing support for the framework, which rests on the external VQA benchmarks. Under the hard rule requiring a specific reduction, this does not qualify as circularity; it is a correctness/robustness concern.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central claim rests on a set of hand-set hyperparameters (loss weights, temperatures, architecture choices) and domain assumptions about distillation and cross-modal grounding. The most consequential, though not a numeric parameter, is the undisclosed split of GQA used for pretraining versus evaluation; that is an assumption that determines whether the headline GQA score is legitimate.

free parameters (4)
  • lambda_1, lambda_2 (loss weights) = not reported
    Defined in Eq. 3 as the balance between contrastive and distillation losses, but the paper never gives their values. These choices affect the learned BreezeCLIP representations and downstream VQA accuracy.
  • tau (InfoNCE temperature) = 0.07
    Temperature in Eq. 1, chosen by hand. Standard value, but it shapes the contrastive training signal.
  • alpha, beta (normalization factors) = 0.5, 0.5
    Appear in Eqs. 1 and 2 to balance directional contrastive terms and distillation terms. Hand-set.
  • BreezeCLIP architecture hyperparameters = 9 transformer layers (2/4/3), embedding dims 512/768
    Manually selected design choices for the compact encoder. The paper states them without sensitivity analysis.
axioms (4)
  • domain assumption Image-text pairs derived from GQA scene graphs are sufficient training data for learning CLIP-style vision-language alignment.
    Section 3.1 constructs pairs by converting scene graphs into language descriptions. The validity of the whole BreezeCLIP pretraining rests on this without supporting evidence.
  • domain assumption Distilling from a frozen CLIP ViT-B/32 teacher transfers useful cross-modal alignment to a much smaller student.
    Used in Eq. 2 and Section 3.1. The paper assumes teacher embeddings are a reliable target, but the teacher itself only reaches 71% cosine similarity on these graph-generated pairs.
  • domain assumption A text-only LLaMA-3.2-1B decoder can learn to ground language in visual pseudo-tokens after light fine-tuning.
    Section 3.3 concatenates visual pseudo-tokens with text tokens and trains with CE loss on responses. The model's VQA performance depends on this transfer working, but no analysis supports it other than the final benchmark numbers.
  • standard math The standard training recipe (Adam/AdamW, StepLR, gradient clipping) converges to a good optimum for this architecture.
    Conventional optimization assumptions invoked without proof in Section 4.2.

pith-pipeline@v1.3.0-alltime-deepseek · 11379 in / 11397 out tokens · 118834 ms · 2026-08-04T20:14:59.687506+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of BcQLM: Efficient Vision-Language Understanding with Distilled Q-Gated Cross-Modal Fusion." pith.science (2026). https://pith.science/paper/365B6D4E

@misc{pith2026250908715,
  author       = {Pith},
  title        = {Pith review of: BcQLM: Efficient Vision-Language Understanding with Distilled Q-Gated Cross-Modal Fusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/365B6D4E}},
  note         = {Machine review of arXiv:2509.08715}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

As multimodal large language models (MLLMs) advance, their large-scale architectures pose challenges for deployment in resource-constrained environments. In the age of large models, where energy efficiency, computational scalability and environmental sustainability are paramount, the development of lightweight and high-performance models is critical for real-world applications. As such, we propose a lightweight MLLM framework for end-to-end visual question answering. Our proposed approach centres on BreezeCLIP, a compact yet powerful vision-language encoder optimised for efficient multimodal understanding. With only 1.2 billion parameters overall, our model significantly reduces computational cost while achieving performance comparable to standard-size MLLMs. Experiments conducted on multiple datasets further validate its effectiveness in balancing accuracy and efficiency. The modular and extensible design enables generalisation to broader multimodal tasks. The proposed lightweight vision-language framework is denoted as BcQLM (BreezeCLIP-enhanced Q-Gated Multimodal Language Model). It offers a promising path toward deployable MLLMs under practical hardware constraints. The source code is available at https://github.com/thico0224/BcQLM.

Figures

Figures reproduced from arXiv: 2509.08715 by Amir Atapour-Abarghouei, Shuang Chen, Sike Xiang.

Figure 1
Figure 1. Figure 1: An overview of the proposed framework. Training consists of two stages: (1) BreezeCLIP encoder is [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: 3D projection of vision-language embeddings [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 2
Figure 2. Figure 2: Cosine similarity in BreezeCLIP distillation. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Example BcQLM responses to GQA questions: “Is there a fork in the bowl?”, “What color is the fur?”,”Is [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 14 linked inside Pith

  1. [1]

    Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. 2018. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2018

  2. [2]

    Lawrence Zitnick, and Devi Parikh

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question answering. In Proceedings of the IEEE International Conference on Computer Vision (ICCV)

    2015

  3. [3]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. https://arxiv.org/abs/2308.12966 Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond . Preprint, arXiv:2308.12966

    Pith/arXiv arXiv 2023

  4. [4]

    Yu, and Qingsong Wen

    Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jinheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S. Yu, and Qingsong Wen. 2025. https://arxiv.org/abs/2503.11733 Llm agents for education: Advances and applications . Preprint, arXiv:2503.11733

    arXiv 2025

  5. [5]

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023. Instructblip: towards general-purpose vision-language models with instruction tuning. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA. Curran Associates Inc

    2023

  6. [6]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long a...

    doi:10.18653/v1/n19-1423 2019

  7. [7]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. https://arxiv.org/abs/2010.11929 An image is worth 16x16 words: Transformers for image recognition at scale . ICLR

    Pith/arXiv arXiv 2021

  8. [8]

    Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, and 3 others. 2023. https://arxiv.org/abs/2303.03378 Palm-e: An e...

    Pith/arXiv arXiv 2023

  9. [9]

    Akira Fukui, Dong Huk Park, Daylen Yang, Anna Rohrbach, Trevor Darrell, and Marcus Rohrbach. 2016. https://doi.org/10.18653/v1/D16-1044 Multimodal compact bilinear pooling for visual question answering and visual grounding . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 457--468, Austin, Texas. Associatio...

    doi:10.18653/v1/d16-1044 2016

  10. [10]

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. 2023. https://doi.org/10.1007/s11263-023-01891-x Clip-adapter: Better vision-language models with feature adapters . Int. J. Comput. Vision, 132(2):581–595

    doi:10.1007/s11263-023-01891-x 2023

  11. [11]

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. https://doi.org/10.1109/CVPR.2017.670 Making the v in vqa matter: Elevating the role of image understanding in visual question answering . In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6325--6334

    doi:10.1109/cvpr.2017.670 2017

  12. [12]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 others. 2024. https://arxiv.org/abs/2407.21783 The llama 3...

    Pith/arXiv arXiv 2024

  13. [13]

    Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P

    Danna Gurari, Qing Li, Abigale J. Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P. Bigham. 2018. https://doi.org/10.1109/CVPR.2018.00380 Vizwiz grand challenge: Answering visual questions from blind people . In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3608--3617

    arXiv 2018

  14. [14]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lo RA : Low-rank adaptation of large language models . In International Conference on Learning Representations

    2022

  15. [15]

    Hudson and Christopher D

    Drew A. Hudson and Christopher D. Manning. 2019. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2019

  16. [16]

    Jin-Hwa Kim, Jaehyun Jun, and Byoung-Tak Zhang. 2018. Bilinear attention networks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS'18, page 1571–1581, Red Hook, NY, USA. Curran Associates Inc

    2018

  17. [17]

    Wonjae Kim, Bokyung Son, and Ildoo Kim. 2021. http://proceedings.mlr.press/v139/kim21k.html Vilt: Vision-and-language transformer without convolution or region supervision . In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 5583--5594. PMLR

    2021

  18. [18]

    Pranath Reddy Kumbam and Kshitij Maruti Vejre. 2024. https://arxiv.org/abs/2401.15501 Floodlense: A framework for chatgpt-based real-time flood detection . Preprint, arXiv:2401.15501

    Pith/arXiv arXiv 2024

  19. [19]

    Rush, Douwe Kiela, Matthieu Cord, and Victor Sanh

    Hugo Laurençon, Lucile Saulnier, Léo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth Karamcheti, Alexander M. Rush, Douwe Kiela, Matthieu Cord, and Victor Sanh. 2023. https://arxiv.org/abs/2306.16527 Obelics: An open web-scale filtered dataset of interleaved image-text documents . Preprint, arXiv:2306.16527

    Pith/arXiv arXiv 2023

  20. [20]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024. https://arxiv.org/abs/2408.03326 Llava-onevision: Easy visual task transfer . Preprint, arXiv:2408.03326

    Pith/arXiv arXiv 2024

  21. [21]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. https://proceedings.mlr.press/v202/li23q.html BLIP -2: Bootstrapping language-image pre-training with frozen image encoders and large language models . In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 19730--19...

    2023

  22. [22]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML

    2022

  23. [23]

    Selvaraju, Akhilesh Deepak Gotmare, Shafiq Joty, Caiming Xiong, and Steven Hoi

    Junnan Li, Ramprasaath R. Selvaraju, Akhilesh Deepak Gotmare, Shafiq Joty, Caiming Xiong, and Steven Hoi. 2021. Align before fuse: Vision and language representation learning with momentum distillation. In NeurIPS

    2021

  24. [24]

    Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. 2024. https://arxiv.org/abs/2312.07533 Vila: On pre-training for visual language models . In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26689--26699

    Pith/arXiv arXiv 2024

  25. [25]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. https://doi.org/10.1109/CVPR52733.2024.02484 Improved baselines with visual instruction tuning . In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 26286--26296

    arXiv 2024

  26. [26]

    Zheng Liu, Hao Liang, Bozhou Li, Tianyi Bai, Wentao Xiong, Chong Chen, Conghui He, Wentao Zhang, and Bin Cui. 2025. https://arxiv.org/abs/2407.20756 Synthvlm: High-efficiency and high-quality synthetic data for vision language models . Preprint, arXiv:2407.20756

    Pith/arXiv arXiv 2025

  27. [27]

    Sachin Mehta and Mohammad Rastegari. 2022. Mobilevit: Light-weight, general-purpose, and mobile-friendly vision transformer. In International Conference on Learning Representations

    2022

  28. [28]

    Advait Parulekar, Liam Collins, Karthikeyan Shanmugam, Aryan Mokhtari, and Sanjay Shakkottai. 2023. https://arxiv.org/abs/2302.07920 Infonce loss provably learns cluster-preserving representations . Preprint, arXiv:2302.07920

    Pith/arXiv arXiv 2023

  29. [29]

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. https://arxiv.org/abs/2412.15115 Qwen2.5 technical report . Preprint, arXiv:2412.15115

    Pith/arXiv arXiv 2025

  30. [30]

    Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever

    Alec Radford, Jong Wook Kim, Chris Hallacy, A. Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. https://arxiv.org/abs/2103.00020 Learning transferable visual models from natural language supervision . In ICML

    Pith/arXiv arXiv 2021

  31. [31]

    Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. https://doi.org/10.1109/CVPR.2018.00474 Mobilenetv2: Inverted residuals and linear bottlenecks . In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4510--4520

    arXiv 2018

  32. [32]

    Zhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu, Yiming Yang, and Denny Zhou. 2020. https://doi.org/10.18653/v1/2020.acl-main.195 M obile BERT : a compact task-agnostic BERT for resource-limited devices . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2158--2170, Online. Association for Computational Li...

    doi:10.18653/v1/2020.acl-main.195 2020

  33. [33]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, and 197 others. 2025. https://arxiv.org/abs/2503.19786...

    Pith/arXiv arXiv 2025

  34. [34]

    Sheng Wang, Zihao Zhao, Xi Ouyang, Qian Wang, and Dinggang Shen. 2023. https://arxiv.org/abs/2302.07257 Chatcad: Interactive computer-aided diagnosis on medical image using large language models . Preprint, arXiv:2302.07257

    Pith/arXiv arXiv 2023

  35. [35]

    Xiaojie Wang, Beibei Wang, Yu Wu, Zhaolong Ning, Song Guo, and Fei Richard Yu. 2024. https://doi.org/10.1109/COMST.2024.3446585 A survey on trustworthy edge intelligence: From security and reliability to transparency and sustainability . IEEE Communications Surveys & Tutorials, pages 1--1

    arXiv 2024

  36. [36]

    Rui Xiao, Sanghwan Kim, Mariana-Iuliana Georgescu, Zeynep Akata, and Stephan Alaniz. 2025. Flair: Vlm with fine-grained language-informed image representations. CVPR

    2025

  37. [37]

    Pengchuan Zhang, Xiujun Li, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, and Jianfeng Gao. 2021. https://doi.org/10.1109/CVPR46437.2021.00553 Vinvl: Revisiting visual representations in vision-language models . In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5575--5584

    arXiv 2021

  38. [38]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  39. [39]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...