Pith. sign in

REVIEW 4 cited by

Similarity-Aware Token Pruning: Your VLM but Faster

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2503.11549 v1 pith:SZO3DK57 submitted 2025-03-14 cs.CV

Ahmadreza Jeddi , Negin Baghbanzadeh , Elham Dolatabadi , Babak Taati This is my paper · ORCID
classification cs.CV
keywords pruningtokensaintvitsvlmsacrossdynamicallyframework
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The computational demands of Vision Transformers (ViTs) and Vision-Language Models (VLMs) remain a significant challenge due to the quadratic complexity of self-attention. While token pruning offers a promising solution, existing methods often introduce training overhead or fail to adapt dynamically across layers. We present SAINT, a training-free token pruning framework that leverages token similarity and a graph-based formulation to dynamically optimize pruning rates and redundancy thresholds. Through systematic analysis, we identify a universal three-stage token evolution process (aligner-explorer-aggregator) in transformers, enabling aggressive pruning in early stages without sacrificing critical information. For ViTs, SAINT doubles the throughput of ViT-H/14 at 224px with only 0.6% accuracy loss on ImageNet-1K, surpassing the closest competitor by 0.8%. For VLMs, we apply SAINT in three modes: ViT-only, LLM-only, and hybrid. SAINT reduces LLaVA-13B's tokens by 75%, achieving latency comparable to LLaVA-7B with less than 1% performance loss across benchmarks. Our work establishes a unified, practical framework for efficient inference in ViTs and VLMs.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs

    cs.CV 2026-07 conditional novelty 6.0 of 10

    C-PTQ weights quantization error by per-channel Fisher information of the task loss, improving low-bit accuracy of multimodal LLMs by small margins over existing channel-wise scaling methods.

  2. Attention-Free and Lightweight Token Reduction for Efficient Vision-Language Models

    cs.CV 2026-07 conditional novelty 5.0 of 10

    ALTR keeps model accuracy nearly intact while pruning up to ~89% of visual tokens, using entropy ranking plus stride sampling on an MLP consistency signal, without attention maps or pairwise comparisons.

  3. Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMs

    cs.CV 2025-06 conditional novelty 5.0 of 10

    CDPruner prunes visual tokens in MLLMs by maximizing instruction-conditioned diversity via a determinantal point process, preserving accuracy at high reduction ratios.

  4. DiffPrune: differentiable information throttling for token pruning in vision-language models

    cs.CV 2026-08 conditional novelty 4.0 of 10

    DiffPrune replaces Gumbel-Softmax surrogate gradients with a differentiable noise-throttling path for visual token scoring, achieving high accuracy retention under aggressive pruning on three VLM families.

Pith tools