Pith. sign in

REVIEW 4 cited by

LiteVAR: Compressing Visual Autoregressive Modelling with Efficient Attention and Quantization

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2411.17178 v1 pith:UUWF3PEA submitted 2024-11-26 cs.CV

LiteVAR: Compressing Visual Autoregressive Modelling with Efficient Attention and Quantization

Rui Xie , Tianchen Zhao , Zhihang Yuan , Rui Wan , Wenxi Gao , Zhenhua Zhu , Xuefei Ning , Yu Wang This is my paper
classification cs.CV
keywords attentionefficientmodelsperformancereductionvisualautoregressivedeployment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Visual Autoregressive (VAR) has emerged as a promising approach in image generation, offering competitive potential and performance comparable to diffusion-based models. However, current AR-based visual generation models require substantial computational resources, limiting their applicability on resource-constrained devices. To address this issue, we conducted analysis and identified significant redundancy in three dimensions of the VAR model: (1) the attention map, (2) the attention outputs when using classifier free guidance, and (3) the data precision. Correspondingly, we proposed efficient attention mechanism and low-bit quantization method to enhance the efficiency of VAR models while maintaining performance. With negligible performance lost (less than 0.056 FID increase), we could achieve 85.2% reduction in attention computation, 50% reduction in overall memory and 1.5x latency reduction. To ensure deployment feasibility, we developed efficient training-free compression techniques and analyze the deployment feasibility and efficiency gain of each technique.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. HACK++: Towards More Effective Head-Aware Key-Value Compression for Efficient Visual Autoregressive Modeling

    cs.CV 2026-06 unverdicted novelty 7.0

    HACK++ is a head-aware KV cache compression framework for VAR models that decouples current-scale attention from historical cache under adaptive per-head budgets to achieve near-lossless generation at 30% attention an...

  2. Token Radius Attention for Efficient Video Generation

    cs.CV 2026-08 conditional novelty 6.0

    Video diffusion transformers can run ~1.5-2x faster with competitive quality by converting each query's attention entropy into a spatially decayed retention radius instead of dense attention.

  3. Adaptive Visual Autoregressive Acceleration via Dual-Linkage Entropy Analysis

    cs.CV 2026-02 conditional novelty 5.0

    A training-free entropy-guided token-pruning framework accelerates VAR image generation up to 2.9× with negligible benchmark loss by activating pruning at an adaptive entropy-growth inflection point and adjusting rati...

  4. FasterVAR: Plug-and-Play Acceleration for Visual Autoregressive Models

    cs.CV 2025-12 conditional novelty 5.0

    Stage-aware pruning of late generation steps, using random projection and cached-feature restoration, speeds up VAR text-to-image models by up to 3.4x with minimal quality loss.