Optimizing Vision-Language Interactions Through Decoder-Only Models

Benjamin Tan; Brian Wong; Kaito Tanaka

REVIEW 4 major objections 4 minor 20 references

Optimizing Vision-Language Interactions Through Decoder-Only Models

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MUDAIF, a decoder-only model that turns images into pseudo-text tokens with a Vision-Token Adapter, claims to outperform encoder-based VLMs across VQA, captioning, and reasoning benchmarks.

desk verdict Eq. (5) doesn't type-check unless N equals L, so the reported SOTA numbers are unattached to a buildable model. read the letter →

arxiv 2412.10758 v1 pith:JWAMCQKA submitted 2024-12-14 cs.CV

Kaito Tanaka , Benjamin Tan , Brian Wong This is my paper · ORCID

classification cs.CV

keywords vision-languagemodeldecoder-onlyarchitectureencoder-freeVLMvision-tokenadapteradaptiveco-attentionvisualquestionansweringimagecaptioningmultimodalreasoning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MUDAIF is a vision-language model built entirely from a decoder Transformer, with no separate image encoder. The paper's central claim is that converting images into pseudo-text tokens through a Vision-Token Adapter, combined with a bidirectional adaptive co-attention mechanism, aligns visual and textual representations better than encoder-based designs. On the benchmarks reported, MUDAIF scores 80.3 on VQA-v2, 65.5 on GQA, and 53.7 on VizWiz, and the authors report 20-35 percent lower training time and 1.5x faster inference at high resolution. If these results hold, encoder-free decoder-only architectures would become a credible alternative to the dominant encoder-decoder paradigm for multimodal tasks.

What carries the argument

Vision-Token Adapter (VTA): turns the image I into a sequence V of N pseudo-text tokens via Conv, Linear, and self-attention, so the decoder can consume vision like language. Adaptive Co-Attention: two softmax attention maps, A_vt from visual to text and A_tv from text to visual, combine into Z = A_vt T + A_tv V; this is the mechanism claimed to enable bidirectional cross-modal interaction. The shared decoder then autoregressively generates text conditioned on Z.

What would settle it

Implement the published equations with an image that produces N pseudo-tokens and a text prompt of length L, with N ≠ L: the addition A_vt T + A_tv V is undefined because the two terms have shapes N×d and L×d. Observing that failure, or running the model with the fusion step replaced by a simple concatenation, would directly test whether the reported VQA-v2 80.3 result depends on an actually implemented alignment not described in the paper.

Watch

Extended reading notes

Core claim

The paper argues that the visual encoder is not necessary for state-of-the-art vision-language performance. Instead, a lightweight Vision-Token Adapter (a convolutional embedding, a linear projection, and self-attention) maps the raw image into N pseudo-text tokens, which a shared decoder processes together with text tokens. An adaptive co-attention mechanism computes attention from visual to text and from text to visual, fusing the two sequences into a joint representation Z = A_vt T + A_tv V. Trained on 45M image-text pairs and instruction-tuned, MUDAIF is reported to beat InstructBLIP, LLaVA-1.5, and EVE-7B on VQA-v2, GQA, VizWiz, SEED, and MM-Vet, as well as in human ratings of relevance, coherence, and informativeness.

Load-bearing premise

The load-bearing premise is that the visual and textual token sequences can be fused by the vector addition in Eq. (5); the paper never specifies how token sequences of different lengths are aligned or projected, so if that sum cannot be defined the architecture as written cannot be built.

Editorial extensions

If this is right

If the reported gains are reproducible, encoder-free models can match or exceed encoder-based VLMs without inheriting resolution and aspect-ratio constraints.
The 20-35 percent training-time reduction and 1.5x inference speedup would make decoder-only VLMs more practical for resource-constrained deployment.
The VTA's pseudo-token design suggests that the same pre-trained decoder can absorb new modalities by adding lightweight adapters rather than retraining a full encoder.
Instruction tuning on top of the 45M-pair pretraining would let a single model serve VQA, captioning, and reasoning tasks with one shared parameter set.
If the high cosine similarity (0.82) between visual and textual embeddings is confirmed, the co-attention mechanism directly improves cross-modal alignment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the fusion equation as printed, Z = A_vt T + A_tv V, requires N and L to match or an unstated projection; a reader cannot reproduce the architecture from the paper alone, and the performance claim depends on that missing piece.
Editorial inference: if the VTA is truly task-agnostic, it should transfer to other decoder-only language models without retraining; testing that transfer would separate the adapter's contribution from the decoder's scale.
Editorial inference: the reported efficiency gains are relative to encoder-based baselines; a direct comparison against a same-scale encoder-free model like EVE-7B would clarify how much of the gain comes from removing the encoder versus from the co-attention design.
Editorial inference: extending the pseudo-token mechanism to video frames or audio spectrograms is a natural next step that the paper lists as future work; the same adapter logic could apply with minor modifications.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Quick take: this one should be desk-rejected, and not just for missing details. The central fusion equation, Eq. (5), is ill-defined for unequal token counts, so the method as written cannot be implemented. The stress-test note is correct: Avt T is [N,d], Atv V is [L,d], and adding them requires N=L, which is never stated or justified. And Eq. (3) is circular, defining V in terms of V.

What's genuinely here: the paper correctly motivates encoder-free VLMs and points to EVE as the main predecessor. The VTA is a conv plus linear projection, which is a standard projector; the adaptive co-attention is a bidirectional cross-attention sum. The only architectural delta over EVE is that sum, which is routine. So novelty is modest.

The experiments are bare tables. No training hyperparameters, no evaluation protocol, no error bars, no code, no model release. ViLT is misdescribed as encoder-free and decoder-only when it is neither, and it's cited with the wrong reference. Ablation numbers come without variance. The human evaluation is a Likert table with no rater details or instructions. None of this supports the abstract's "new standard" claim.

The technical blocker is not cosmetic. Eq. (5) is undefined unless N is tied to L, and the paper never says how that tie is handled during inference. A referee could ask for clarification, but the paper gives too little to even start a useful review.

Who gets value from this? Someone skimming the related-work framing might find the encoder-free motivation worth a minute. No serious reader should build on the method as stated, and I wouldn't cite it.

Recommendation: desk reject. If the authors return with a well-defined fusion rule, an actual training setup, and reproducible evaluation, it could be worth another look.

Referee Report

4 major / 4 minor

Summary. The manuscript proposes MUDAIF, a decoder-only vision-language model that removes the separate visual encoder and instead converts image patches into pseudo-text tokens via a Vision-Token Adapter (VTA), then fuses visual and textual tokens with an adaptive co-attention mechanism. The paper reports evaluations on VQA-v2, GQA, VizWiz, SEED, MM-Vet, and a BLEU-based captioning measurement against InstructBLIP, LLaVA-1.5, and EVE-7B, and it claims that MUDAIF consistently outperforms these baselines. Ablation studies, human evaluations, robustness tests, and generalization experiments are also reported. The central contribution is the claim that an encoder-free decoder-only model with a simple adapter and co-attention can match or beat established encoder-based VLMs.

Significance. Encoder-free vision-language models are a timely direction, and the paper identifies a real efficiency and scalability motivation for avoiding a dedicated visual encoder. The proposed VTA and adaptive co-attention mechanisms could in principle be a useful contribution if they were well-defined and rigorously validated. However, the manuscript does not provide an implementable architecture: the key fusion equation is dimensionally ill-typed, and the empirical section consists of bare tables without training details, error bars, code, or evaluation protocols. The central claim of the abstract is therefore not supported by the presented evidence. Credit is due for including an ablation that separates the VTA and co-attention contributions, but this does not compensate for the missing definitions and verifiability.

major comments (4)

[III-C, Eq. (5)] The fusion equation Z = A_vt T + A_tv V is not well-defined for token sequences of different lengths. With A_vt in R^{N x L}, A_tv in R^{L x N}, T in R^{L x d}, and V in R^{N x d}, the products A_vt T and A_tv V have shapes N x d and L x d, respectively, and can be summed only when N = L. The paper never states or justifies this equality, and during autoregressive inference the text length L varies, so the sum cannot be evaluated even if training sequences were padded to a common length. Because this fusion step is the central mechanism of the model, the accuracy numbers in Tables I and II cannot be attributed to the system as described.
[III-B, Eq. (3)] Equation (3) defines V as the output of self-attention while using V as the input value matrix on the right-hand side, making the definition circular. A separate value matrix, e.g., V' = Linear(V), or distinct notation is needed. As written, the VTA cannot be instantiated, and this compounds the shape problem in Eq. (5) because V is a core operand in both equations.
[IV, Tables I-VI] The quantitative claims are presented as bare numbers with no training configuration (model size, optimizer, learning rate, batch size, learning-rate schedule, number of steps), no image preprocessing or prompt-template details, no decoding settings, no error bars or significance tests, and no code or data release. The human evaluation in Table III and the user study in Table VI do not report the number of raters, inter-annotator agreement, or statistical intervals. Without this information, the abstract's claim that MUDAIF 'consistently outperforms state-of-the-art methods' is not verifiable.
[IV-A] Only three baselines are compared, and the paper does not state whether their numbers were reimplemented or taken from the literature, nor does it include standard recent encoder-free baselines used in the EVE paper. The evaluation protocol is also incomplete: captioning is measured by BLEU, but no captioning benchmark is named, and the SEED and MM-Vet composite scores are not defined. This makes the head-to-head comparisons in Tables I, II, IV, and V difficult to interpret.

minor comments (4)

[Throughout] There are typographical inconsistencies in benchmark and model names, such as 'MM-V et' for MM-Vet and 'LLaV A-1.5' for LLaVA-1.5.
[III-D, Eqs. (6)-(7)] The losses are defined only over text tokens t_i, but no masking or attention structure is specified for how V interacts with the decoder's causal masking; a precise description of the token layout would clarify the training objective.
[III-A] The term 'pseudo-text tokens' is introduced without specifying token lengths, embedding initialization, or how these tokens are positioned relative to text tokens in the unified sequence; a concrete example would help.
[I and IV-A] The introduction mentions COCO, LAION, and Visual Genome as training data, but no references or data splits are provided, and the 45M-pair dataset is not described in terms of filtering, deduplication, or train/validation separation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: reported gains are empirical benchmark comparisons, not derivations that reduce to fitted inputs or self-citations.

full rationale

The paper makes an architectural proposal and reports benchmark results; it does not derive a prediction from a fitted parameter or from a prior result by the same authors. The central claim—that MUDAIF outperforms encoder-based and encoder-free baselines on VQA, captioning, and reasoning tasks—is supported by reported accuracy, BLEU, composite scores, ablations, and human ratings, all of which are empirical comparisons rather than mathematical consequences of the model definition. No parameter is fitted to a benchmark subset and then relabeled as a prediction of that same quantity, and no uniqueness theorem or load-bearing claim is imported from the authors' own previous work. The notational problems in Eqs. (3) and (5), where V is reused on both sides of the self-attention equation and where the terms in the co-attention sum have incompatible shapes unless the pseudo-token count equals the text-token count, are genuine implementability defects, but they are correctness and reproducibility flaws rather than circularity: a non-well-defined equation does not make an empirical accuracy number equivalent to its inputs. Absent any fitted-input prediction or self-citation chain, the circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claims depend on unstated loss weights and model dimensions, plus an ill-defined fusion equation. There are no invented physical entities, but the architecture itself rests on ad hoc assumptions about how to mix token sequences.

free parameters (2)

loss weights λ_pretrain, λ_task = not reported
Equation (8) combines pretraining and task losses, but the paper never reports the values used; the central training objective depends on them.
number of pseudo-tokens N and embedding dimension d = not reported
The VTA output V is N×d, but the paper never specifies N or d, which are crucial for the model's capacity and for Eq. (5) to be defined.

assumptions (4)

standard math Standard transformer self-attention and autoregressive language modeling loss are valid.
Equations (3), (6), (7) rely on standard attention and cross-entropy, no derivation needed.
domain assumption A convolutional embedding followed by a linear projection can produce tokens that retain enough visual information for language modeling.
Section III-B defines VTA as Linear(Conv(I)); the paper provides no evidence that this is sufficient, but the whole method depends on it.
ad hoc to paper The fusion output Z = Avt T + Atv V is a mathematically valid way to combine modalities.
Equation (5) is introduced without justification or dimensional analysis; the paper never shows how differing sequence lengths are reconciled.
domain assumption The 45M image-text pairs from COCO, LAION, and Visual Genome were filtered and used as described.
Section IV-A states the training data but gives no preprocessing, filtering, or splits; the results depend on this unverified setup.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Optimizing Vision-Language Interactions Through Decoder-Only Models." pith.science (2026). https://pith.science/paper/JWAMCQKA

@misc{pith2026241210758,
  author       = {Pith},
  title        = {Pith review of: Optimizing Vision-Language Interactions Through Decoder-Only Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JWAMCQKA}},
  note         = {Machine review of arXiv:2412.10758}
}

read the original abstract

Vision-Language Models (VLMs) have emerged as key enablers for multimodal tasks, but their reliance on separate visual encoders introduces challenges in efficiency, scalability, and modality alignment. To address these limitations, we propose MUDAIF (Multimodal Unified Decoder with Adaptive Input Fusion), a decoder-only vision-language model that seamlessly integrates visual and textual inputs through a novel Vision-Token Adapter (VTA) and adaptive co-attention mechanism. By eliminating the need for a visual encoder, MUDAIF achieves enhanced efficiency, flexibility, and cross-modal understanding. Trained on a large-scale dataset of 45M image-text pairs, MUDAIF consistently outperforms state-of-the-art methods across multiple benchmarks, including VQA, image captioning, and multimodal reasoning tasks. Extensive analyses and human evaluations demonstrate MUDAIF's robustness, generalization capabilities, and practical usability, establishing it as a new standard in encoder-free vision-language models.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 4 canonical work pages

[1]

Unveili ng encoder-free vision-language models,

H. Diao, Y . Cui, X. Li, Y . Wang, H. Lu, and X. Wang, “Unveili ng encoder-free vision-language models,” CoRR, vol. abs/2406.11832,

arXiv
[2]

Rethinking visual de pendency in long-context reasoning for large vision-language models,

Y . Zhou, Z. Rao, J. Wan, and J. Shen, “Rethinking visual de pendency in long-context reasoning for large vision-language models, ” arXiv preprint arXiv:2410.19732, 2024

arXiv 2024
[3]

Visual in-context le arning for large vision-language models,

Y . Zhou, X. Li, Q. Wang, and J. Shen, “Visual in-context le arning for large vision-language models,” in Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand an d virtual meeting, August 11-16, 2024 . Association for Computational Linguis- tics, 2024, pp. 15 890–15 902

2024
[4]

Claret: P re-training a correlation-aware context-to-event transformer for eve nt-centric gener- ation and classiﬁcation,

Y . Zhou, T. Shen, X. Geng, G. Long, and D. Jiang, “Claret: P re-training a correlation-aware context-to-event transformer for eve nt-centric gener- ation and classiﬁcation,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (V olume 1: L ong Papers), 2022, pp. 2559–2575

work page 2022
[5]

Eventber t: A pre- trained model for event correlation reasoning,

Y . Zhou, X. Geng, T. Shen, G. Long, and D. Jiang, “Eventber t: A pre- trained model for event correlation reasoning,” in Proceedings of the ACM W eb Conference 2022, 2022, pp. 850–859

2022
[6]

Visionllm: Large language mode l is also an open-ended decoder for vision-centric tasks,

W. Wang, Z. Chen, X. Chen, J. Wu, X. Zhu, G. Zeng, P . Luo, T. L u, J. Zhou, Y . Qiao, and J. Dai, “Visionllm: Large language mode l is also an open-ended decoder for vision-centric tasks,” in Advances in Neural Information Processing Systems 36: Annual Conference on Ne ural Information Processing Systems 2023, NeurIPS 2023, New Orl eans, LA, USA, Decembe...

work page 2023
[7]

Visionllm v2: An end-to-end generalist multimodal large language model for hundreds of vision-language tasks,

J. Wu, M. Zhong, S. Xing, Z. Lai, Z. Liu, W. Wang, Z. Chen, X. Zhu, L. Lu, T. Lu, P . Luo, Y . Qiao, and J. Dai, “Visionllm v2: An end-to-end generalist multimodal large language model for hundreds of vision-language tasks,” CoRR, vol. abs/2406.08394, 2024. [Online]. Available: https://doi.org/10.48550/arXiv.2406.08394

arXiv doi:10.48550/arxiv.2406.08394 2024
[8]

Moe-llava: Mixture of experts for large vision -language models,

B. Lin, Z. Tang, Y . Y e, J. Cui, B. Zhu, P . Jin, J. Zhang, M. Ni ng, and L. Y uan, “Moe-llava: Mixture of experts for large vision -language models,” CoRR, vol. abs/2401.15947, 2024. [Online]. Available: https://doi.org/10.48550/arXiv.2401.15947

arXiv doi:10.48550/arxiv.2401.15947 2024

Show all 20 references

[9]

Enhancing large vision language models with self- training on image comprehension,

Y . Deng, P . Lu, F. Yin, Z. Hu, S. Shen, J. Zou, K. Chang, and W. Wang, “Enhancing large vision language models with self- training on image comprehension,” CoRR, vol. abs/2405.19716, 2024. [Online]. Available: https://doi.org/10.48550/arXiv.2405.19716

2024 doi arXiv
[10]

Fine-tuning large vision-language models as decision-making agents via rein forcement learning,

Y . Zhai, H. Bai, Z. Lin, J. Pan, S. Tong, Y . Zhou, A. Suhr, S. Xie, Y . LeCun, Y . Ma, and S. Levine, “Fine-tuning large vision-language models as decision-making agents via rein forcement learning,” CoRR, vol. abs/2405.10292, 2024. [Online]. Available: https://doi.org/10.48...

2024 doi arXiv
[11]

Triple sequence generati ve adversarial nets for unsupervised image captioning,

Y . Zhou, W. Tao, and W. Zhang, “Triple sequence generati ve adversarial nets for unsupervised image captioning,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal P rocessing (ICASSP). IEEE, 2021, pp. 7598–7602

2021
[12]

Sketch storytelling,

Y . Zhou, “Sketch storytelling,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICA SSP). IEEE, 2022, pp. 4748–4752

2022
[13]

Are we on the right way fo r evaluating large vision-language models?

L. Chen, J. Li, X. Dong, P . Zhang, Y . Zang, Z. Chen, H. Duan , J. Wang, Y . Qiao, D. Lin, and F. Zhao, “Are we on the right way fo r evaluating large vision-language models?” CoRR, vol. abs/2403.20330,

arXiv
[14]

Internvl: Scaling up vision foundation models and alignin g for generic visual-linguistic tasks,

Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, M. Zhong, Q. Zhang, X. Zhu, L. Lu, B. Li, P . Luo, T. Lu, Y . Qiao, and J. Dai, “Internvl: Scaling up vision foundation models and alignin g for generic visual-linguistic tasks,” CoRR, vol. abs/2312.14238, 2023. [Online]. Avail...

2023 doi arXiv
[15]

Available: https://doi.org/10.48550/ar Xiv.2403.20330

[Online]. Available: https://doi.org/10.48550/ar Xiv.2403.20330

doi arXiv
[16]

Multimodal event transformer for i mage-guided story ending generation,

Y . Zhou and G. Long, “Multimodal event transformer for i mage-guided story ending generation,” in Proceedings of the 17th Conference of the European Chapter of the Association for Computational Ling uistics, 2023, pp. 3434–3444

2023
[17]

Texthawk2: A large vi sion- language model excels in bilingual OCR and grounding with 16 x fewer tokens,

Y . Y u, M. Liao, J. Zhang, and J. Wu, “Texthawk2: A large vi sion- language model excels in bilingual OCR and grounding with 16 x fewer tokens,” CoRR, vol. abs/2410.05261, 2024. [Online]. Available: https://doi.org/10.48550/arXiv.2410.05261

2024 doi arXiv
[18]

Vilt: Vision-and-language t ransformer without convolution or region supervision,

W. Kim, B. Son, and I. Kim, “Vilt: Vision-and-language t ransformer without convolution or region supervision,” in Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 1 8-24 July 2021, Virtual Event , ser. Proceedings of Machine Learning Research, ...

2021
[19]

Style-aware contrastive learning for multi-styl e image caption- ing,

——, “Style-aware contrastive learning for multi-styl e image caption- ing,” in Findings of the Association for Computational Linguistics : EACL 2023 , 2023, pp. 2257–2267

2023
[2024]

Available: https://doi.org/10.48550/ar Xiv.2406.11832

[Online]. Available: https://doi.org/10.48550/ar Xiv.2406.11832

doi arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Unveili ng encoder-free vision-language models,

H. Diao, Y . Cui, X. Li, Y . Wang, H. Lu, and X. Wang, “Unveili ng encoder-free vision-language models,” CoRR, vol. abs/2406.11832,

arXiv

[2] [2]

Rethinking visual de pendency in long-context reasoning for large vision-language models,

Y . Zhou, Z. Rao, J. Wan, and J. Shen, “Rethinking visual de pendency in long-context reasoning for large vision-language models, ” arXiv preprint arXiv:2410.19732, 2024

arXiv 2024

[3] [3]

Visual in-context le arning for large vision-language models,

Y . Zhou, X. Li, Q. Wang, and J. Shen, “Visual in-context le arning for large vision-language models,” in Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand an d virtual meeting, August 11-16, 2024 . Association for Computational Linguis- tics, 2024, pp. 15 890–15 902

2024

[4] [4]

Claret: P re-training a correlation-aware context-to-event transformer for eve nt-centric gener- ation and classiﬁcation,

Y . Zhou, T. Shen, X. Geng, G. Long, and D. Jiang, “Claret: P re-training a correlation-aware context-to-event transformer for eve nt-centric gener- ation and classiﬁcation,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (V olume 1: L ong Papers), 2022, pp. 2559–2575

work page 2022

[5] [5]

Eventber t: A pre- trained model for event correlation reasoning,

Y . Zhou, X. Geng, T. Shen, G. Long, and D. Jiang, “Eventber t: A pre- trained model for event correlation reasoning,” in Proceedings of the ACM W eb Conference 2022, 2022, pp. 850–859

2022

[6] [6]

Visionllm: Large language mode l is also an open-ended decoder for vision-centric tasks,

W. Wang, Z. Chen, X. Chen, J. Wu, X. Zhu, G. Zeng, P . Luo, T. L u, J. Zhou, Y . Qiao, and J. Dai, “Visionllm: Large language mode l is also an open-ended decoder for vision-centric tasks,” in Advances in Neural Information Processing Systems 36: Annual Conference on Ne ural Information Processing Systems 2023, NeurIPS 2023, New Orl eans, LA, USA, Decembe...

work page 2023

[7] [7]

Visionllm v2: An end-to-end generalist multimodal large language model for hundreds of vision-language tasks,

J. Wu, M. Zhong, S. Xing, Z. Lai, Z. Liu, W. Wang, Z. Chen, X. Zhu, L. Lu, T. Lu, P . Luo, Y . Qiao, and J. Dai, “Visionllm v2: An end-to-end generalist multimodal large language model for hundreds of vision-language tasks,” CoRR, vol. abs/2406.08394, 2024. [Online]. Available: https://doi.org/10.48550/arXiv.2406.08394

arXiv doi:10.48550/arxiv.2406.08394 2024

[8] [8]

Moe-llava: Mixture of experts for large vision -language models,

B. Lin, Z. Tang, Y . Y e, J. Cui, B. Zhu, P . Jin, J. Zhang, M. Ni ng, and L. Y uan, “Moe-llava: Mixture of experts for large vision -language models,” CoRR, vol. abs/2401.15947, 2024. [Online]. Available: https://doi.org/10.48550/arXiv.2401.15947

arXiv doi:10.48550/arxiv.2401.15947 2024