FlashSVD fuses low-rank SVD projections into attention and feed-forward GPU kernels so SVD-compressed transformers avoid materializing dense activations, cutting activation memory at a latency cost.
MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We present MorphNet, an approach to automate the design of neural network structures. MorphNet iteratively shrinks and expands a network, shrinking via a resource-weighted sparsifying regularizer on activations and expanding via a uniform multiplicative factor on all layers. In contrast to previous approaches, our method is scalable to large networks, adaptable to specific resource constraints (e.g. the number of floating-point operations per inference), and capable of increasing the network's performance. When applied to standard network architectures on a wide variety of datasets, our approach discovers novel structures in each domain, obtaining higher performance while respecting the resource constraint.
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models
FlashSVD fuses low-rank SVD projections into attention and feed-forward GPU kernels so SVD-compressed transformers avoid materializing dense activations, cutting activation memory at a latency cost.