Pith. sign in

REVIEW 15 cited by

LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.18139 v1 pith:7QFYZ2PX submitted 2024-06-26 cs.CL cs.CV

Zhongwei Wan , Ziang Wu , Che Liu , Jinfa Huang , Zhihong Zhu , Peng Jin , Longyue Wang , Li Yuan This is my paper · ORCID
classification cs.CLcs.CV
keywords cachemultimodallong-contextimagelook-mtextualcontextsinference
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Long-context Multimodal Large Language Models (MLLMs) demand substantial computational resources for inference as the growth of their multimodal Key-Value (KV) cache, in response to increasing input lengths, challenges memory and time efficiency. Unlike single-modality LLMs that manage only textual contexts, the KV cache of long-context MLLMs includes representations from multiple images with temporal and spatial relationships and related textual contexts. The predominance of image tokens means traditional optimizations for LLMs' KV caches are unsuitable for multimodal long-context settings, and no prior works have addressed this challenge. In this work, we introduce LOOK-M, a pioneering, fine-tuning-free approach that efficiently reduces the multimodal KV cache size while maintaining performance comparable to a full cache. We observe that during prompt prefill, the model prioritizes more textual attention over image features, and based on the multimodal interaction observation, a new proposed text-prior method is explored to compress the KV cache. Furthermore, to mitigate the degradation of image contextual information, we propose several compensatory strategies using KV pairs merging. LOOK-M demonstrates that with a significant reduction in KV Cache memory usage, such as reducing it by 80% in some cases, it not only achieves up to 1.5x faster decoding but also maintains or even enhances performance across a variety of long context multimodal tasks.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 15 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PhyCheck: Fine-Grained Evidence-Grounded Dataset for Physical Law Understanding in Video-LLMs

    cs.CV 2026-08 conditional novelty 7.0 of 10

    PhyCheck is a 69,825-pair video QA benchmark that tests and improves Video-LLMs' ability to judge whether events obey physical laws, with fine-grained evidence questions and a context-sensitivity pilot.

  2. What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agents

    cs.LG 2026-07 conditional novelty 6.0 of 10

    KV-cache eviction, prompt compression, recurrent state bounding, and agent memory consolidation are unified as one rate-distortion problem with a shared lower bound, shared failure mode, and transferable mechanisms.

  3. Multi-Granular Spatio-Temporal Token Merging for Training-Free Acceleration of Video LLMs

    cs.CV 2025-07 conditional novelty 6.0 of 10

    STTM is a training-free token merging method that reduces video LLM token counts by 50-70% while keeping accuracy nearly unchanged, yielding 2-3x speedups on six video QA benchmarks.

  4. MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference

    cs.LG 2025-06 conditional novelty 6.0 of 10

    MadaKV adaptively splits the KV cache budget by attention-head modality preference and compensates across layers, cutting cache memory by 80-95% and speeding decoding by 1.3-1.5x with small accuracy loss.

  5. Zooming from Context to Cue: Hierarchical Preference Optimization for Multi-Image MLLMs

    cs.CV 2025-05 conditional novelty 6.0 of 10

    Context-to-Cue Direct Preference Optimization (CcDPO) reduces multi-image hallucinations in 7B multimodal LLMs by training on perturbed full-sequence captions and region-focused visual prompts, improving average multi...

  6. Streamline Without Sacrifice -- Squeeze out Computation Redundancy in LMM

    cs.CV 2025-05 conditional novelty 6.0 of 10

    ProxyV introduces proxy vision tokens that take over expensive attention and feed-forward computation in later layers of decoder-only multimodal models, cutting FLOPs by 25-46% while retaining or improving accuracy on...

  7. Speculative Prefill: Turbocharging TTFT with Lightweight and Training-Free Token Importance Estimation

    cs.CL 2025-02 conditional novelty 6.0 of 10

    A training-free method that selects a subset of prompt tokens with a small speculator model to accelerate LLM prefill, yielding up to 7.66x TTFT speedup on Llama-3.1-405B.

  8. AKVQ-VL: Attention-Aware KV Cache Adaptive 2-Bit Quantization for Vision-Language Models

    cs.CL 2025-01 conditional novelty 6.0 of 10

    AKVQ-VL quantizes VLM KV caches to mostly 2 bits with attention-aware token protection and Walsh-Hadamard outlier removal, staying near FP16 accuracy on MileBench.

  9. Feather the Throttle: Revisiting Visual Token Pruning for Vision-Language Model Acceleration

    cs.CV 2024-12 conditional novelty 6.0 of 10

    FastV's early image-token pruning drops most top-of-image tokens because of RoPE's positional decay, most VQA benchmarks cannot even detect this flaw, and the FEATHER method fixes it for localization.

  10. DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Models

    cs.CV 2024-11 conditional novelty 6.0 of 10

    A training-free two-stage token compression method speeds up video language model inference by about 1.5x and cuts memory use, while keeping accuracy close to the uncompressed model.

  11. MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models

    cs.AI 2026-06 conditional novelty 5.0 of 10

    By sampling variance-inflated query vectors during prefilling, MM-ShiftKV selects prompt KV caches that better match decoding-time attention and outperforms prior prefill-only KV compression on multimodal benchmarks a...

  12. Breaking the Boundaries of Long-Context LLM Inference: Adaptive KV Management on a Single Commodity GPU

    cs.OS 2025-06 conditional novelty 5.0 of 10

    LeoAM reports a 3.46x average latency speedup for long-context LLM inference on one commodity GPU by adaptively chunking KV data and loading compact key abstracts from disk instead of full KV values.

  13. LaVi: Efficient Large Vision-Language Models via Internal Feature Modulation

    cs.CV 2025-06 conditional novelty 5.0 of 10

    LaVi encodes visual context into LayerNorm affine parameters, bypassing visual token concatenation, and reports LLaVA-comparable accuracy at a 94% FLOP reduction.

  14. Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression

    cs.LG 2025-05 conditional novelty 5.0 of 10

    ScaleKV cuts KV cache memory for Visual Autoregressive text-to-image generation to 10% by classifying layers as drafters or refiners per scale and pruning low-attention tokens while keeping benchmark scores nearly unchanged.

  15. LinVT: Empower Your Image-level Large Language Model to Understand Videos

    cs.CV 2024-12 conditional novelty 5.0 of 10

    A plug-and-play linear video tokenizer converts existing image-based LLMs into video-understanding LLMs by condensing frames into weighted-average tokens while preserving image capabilities.

Pith tools