Pith. sign in

REVIEW 8 cited by

LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.18139 v1 pith:7QFYZ2PX submitted 2024-06-26 cs.CL cs.CV

Zhongwei Wan , Ziang Wu , Che Liu , Jinfa Huang , Zhihong Zhu , Peng Jin , Longyue Wang , Li Yuan This is my paper
classification cs.CLcs.CV
keywords cachemultimodallong-contextimagelook-mtextualcontextsinference
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Long-context Multimodal Large Language Models (MLLMs) demand substantial computational resources for inference as the growth of their multimodal Key-Value (KV) cache, in response to increasing input lengths, challenges memory and time efficiency. Unlike single-modality LLMs that manage only textual contexts, the KV cache of long-context MLLMs includes representations from multiple images with temporal and spatial relationships and related textual contexts. The predominance of image tokens means traditional optimizations for LLMs' KV caches are unsuitable for multimodal long-context settings, and no prior works have addressed this challenge. In this work, we introduce LOOK-M, a pioneering, fine-tuning-free approach that efficiently reduces the multimodal KV cache size while maintaining performance comparable to a full cache. We observe that during prompt prefill, the model prioritizes more textual attention over image features, and based on the multimodal interaction observation, a new proposed text-prior method is explored to compress the KV cache. Furthermore, to mitigate the degradation of image contextual information, we propose several compensatory strategies using KV pairs merging. LOOK-M demonstrates that with a significant reduction in KV Cache memory usage, such as reducing it by 80% in some cases, it not only achieves up to 1.5x faster decoding but also maintains or even enhances performance across a variety of long context multimodal tasks.

Discussion (0). Sign in to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PhyCheck: Fine-Grained Evidence-Grounded Dataset for Physical Law Understanding in Video-LLMs

    cs.CV 2026-08 conditional novelty 7.0 of 10

    PhyCheck is a 69,825-pair video QA benchmark that tests and improves Video-LLMs' ability to judge whether events obey physical laws, with fine-grained evidence questions and a context-sensitivity pilot.

  2. What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agents

    cs.LG 2026-07 conditional novelty 6.0 of 10

    KV-cache eviction, prompt compression, recurrent state bounding, and agent memory consolidation are unified as one rate-distortion problem with a shared lower bound, shared failure mode, and transferable mechanisms.

  3. Multi-Granular Spatio-Temporal Token Merging for Training-Free Acceleration of Video LLMs

    cs.CV 2025-07 conditional novelty 6.0 of 10

    STTM is a training-free token merging method that reduces video LLM token counts by 50-70% while keeping accuracy nearly unchanged, yielding 2-3x speedups on six video QA benchmarks.

  4. MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference

    cs.LG 2025-06 conditional novelty 6.0 of 10

    MadaKV adaptively splits the KV cache budget by attention-head modality preference and compensates across layers, cutting cache memory by 80-95% and speeding decoding by 1.3-1.5x with small accuracy loss.

  5. Zooming from Context to Cue: Hierarchical Preference Optimization for Multi-Image MLLMs

    cs.CV 2025-05 conditional novelty 6.0 of 10

    Context-to-Cue Direct Preference Optimization (CcDPO) reduces multi-image hallucinations in 7B multimodal LLMs by training on perturbed full-sequence captions and region-focused visual prompts, improving average multi...

  6. MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models

    cs.AI 2026-06 conditional novelty 5.0 of 10

    By sampling variance-inflated query vectors during prefilling, MM-ShiftKV selects prompt KV caches that better match decoding-time attention and outperforms prior prefill-only KV compression on multimodal benchmarks a...

  7. Breaking the Boundaries of Long-Context LLM Inference: Adaptive KV Management on a Single Commodity GPU

    cs.OS 2025-06 conditional novelty 5.0 of 10

    LeoAM reports a 3.46x average latency speedup for long-context LLM inference on one commodity GPU by adaptively chunking KV data and loading compact key abstracts from disk instead of full KV values.

  8. LaVi: Efficient Large Vision-Language Models via Internal Feature Modulation

    cs.CV 2025-06 conditional novelty 5.0 of 10

    LaVi encodes visual context into LayerNorm affine parameters, bypassing visual token concatenation, and reports LLaVA-comparable accuracy at a 94% FLOP reduction.

Pith tools