MoE structure, not parameter count, governs INT4 robustness in sub-3B VLMs; SigLIP INT8 latency spikes on Jetson Ampere are a BitsAndBytes-Ampere interaction, and INT4 VRAM savings come with TPOT and energy penalties.
Energy consumption generally scales with latency, and accuracy degradation under quantization varies substantially across model architectures and configuration types
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2026 1verdicts
ACCEPT 1representative citing papers
citing papers explorer
-
Rethinking Small VLM Quantization: From Component-Wise Analysis to Hardware-Aware Edge Deployment
MoE structure, not parameter count, governs INT4 robustness in sub-3B VLMs; SigLIP INT8 latency spikes on Jetson Ampere are a BitsAndBytes-Ampere interaction, and INT4 VRAM savings come with TPOT and energy penalties.