Pith. sign in

REVIEW 5 cited by

EVLM: An Efficient Vision-Language Model for Visual Understanding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.14177 v1 pith:QFYC36FK submitted 2024-07-19 cs.CV

Kaibing Chen , Dong Shen , Hanwen Zhong , Huasong Zhong , Kui Xia , Di Xu , Wei Yuan , Yifei Hu
show 9 more authors
Bin Wen Tianke Zhang Changyi Liu Dewen Fan Huihui Xiao Jiahong Wu Fan Yang Size Li Di Zhang
This is my paper · ORCID
classification cs.CV
keywords languagemodelmodelsvisualmulti-modalsignalscaptioningcomputational
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In the field of multi-modal language models, the majority of methods are built on an architecture similar to LLaVA. These models use a single-layer ViT feature as a visual prompt, directly feeding it into the language models alongside textual tokens. However, when dealing with long sequences of visual signals or inputs such as videos, the self-attention mechanism of language models can lead to significant computational overhead. Additionally, using single-layer ViT features makes it challenging for large language models to perceive visual signals fully. This paper proposes an efficient multi-modal language model to minimize computational costs while enabling the model to perceive visual signals as comprehensively as possible. Our method primarily includes: (1) employing cross-attention to image-text interaction similar to Flamingo. (2) utilize hierarchical ViT features. (3) introduce the Mixture of Experts (MoE) mechanism to enhance model effectiveness. Our model achieves competitive scores on public multi-modal benchmarks and performs well in tasks such as image captioning and video captioning.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. RedundancyLens: Revealing and Exploiting Visual Token Processing Redundancy for Efficient Decoder-Only MLLMs

    cs.CV 2025-01 conditional novelty 7.0 of 10

    Decoder-only MLLMs tolerate simplified self-attention and FFN processing for visual tokens in about half of their layers, enabling a training-free FLOPs reduction method.

  2. Ascending the Infinite Ladder: Benchmarking Spatial Deformation Reasoning in Vision-Language Models

    cs.CV 2025-07 conditional novelty 6.0 of 10

    Vision-language models can handle some 2D shape puzzles but nearly all fail at multi-step 3D spatial deformation reasoning.

  3. CoMemo: LVLMs Need Image Context with Image Memory

    cs.CV 2025-06 conditional novelty 6.0 of 10

    CoMemo adds a cross-attention image-memory path and thumbnail-anchored position encoding to reduce visual neglect in long-context and multi-image LVLM tasks.

  4. From One-to-One to Many-to-Many: Dynamic Cross-Layer Injection for Deep Vision-Language Fusion

    cs.CV 2026-01 conditional novelty 5.0 of 10

    CLI injects many ViT layers into many LLM layers via LoRA projectors and gated fusion, yielding modest and inconsistent benchmark gains over LLaVA baselines.

  5. A Survey on Video Temporal Grounding with Multimodal Large Language Model

    cs.CV 2025-08 unverdicted novelty 3.0 of 10

    A taxonomized review of video temporal grounding with multimodal large language models, covering model roles, training paradigms, feature processing, benchmarks, and open problems.

Pith tools