D8-equivariant linear layers give ViTs a 5.33x FLOP reduction and 8x parameter reduction per layer, and hybrid octic ViTs match or slightly exceed ImageNet-1K accuracy while using about 40% fewer FLOPs.
Getting vit in shape: Scaling laws for compute-optimal model design
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CV 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Quick ViTs: Speeding up Vision Transformers through Equivariance
D8-equivariant linear layers give ViTs a 5.33x FLOP reduction and 8x parameter reduction per layer, and hybrid octic ViTs match or slightly exceed ImageNet-1K accuracy while using about 40% fewer FLOPs.