Less is More: A Simple yet Effective Token Reduction Method for Efficient Multi-modal LLMs

Benyou Wang; Dingjie Song; Michael Guan; Shunian Chen; Wenjun Wang; Xidong Wang

Less is More: A Simple yet Effective Token Reduction Method for Efficient Multi-modal LLMs

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2409.10994 v3 pith:NAGBSUOL submitted 2024-09-17 cs.CL cs.AIcs.CVcs.MM

Less is More: A Simple yet Effective Token Reduction Method for Efficient Multi-modal LLMs

Dingjie Song , Wenjun Wang , Shunian Chen , Xidong Wang , Michael Guan , Benyou Wang This is my paper

classification cs.CL cs.AIcs.CVcs.MM

keywords reductionmodelstrimacrossefficientmethodmllmsperformance

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

0 comments

Share X Bluesky LinkedIn Reddit HN

read the original abstract

The rapid advancement of Multimodal Large Language Models (MLLMs) has led to remarkable performances across various domains. However, this progress is accompanied by a substantial surge in the resource consumption of these models. We address this pressing issue by introducing a new approach, Token Reduction using CLIP Metric (TRIM), aimed at improving the efficiency of MLLMs without sacrificing their performance. Inspired by human attention patterns in Visual Question Answering (VQA) tasks, TRIM presents a fresh perspective on the selection and reduction of image tokens. The TRIM method has been extensively tested across 12 datasets, and the results demonstrate a significant reduction in computational overhead while maintaining a consistent level of performance. This research marks a critical stride in efficient MLLM development, promoting greater accessibility and sustainability of high-performing models.

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Decoupled Similarity for Task-Aware Token Pruning in Large Vision-Language Models
cs.CV 2026-04 unverdicted novelty 6.0

DeSAP uses decoupled cross-modal similarity plus visual saliency to prune visual tokens in LVLMs, retaining 11.1% tokens for 10x FLOPs reduction and 98.1% performance on LLaVA-1.5-7B.