Striving for Simplicity: The All Convolutional Net
read the original abstract
Most modern convolutional neural networks (CNNs) used for object recognition are built using the same principles: Alternating convolution and max-pooling layers followed by a small number of fully connected layers. We re-evaluate the state of the art for object recognition from small images with convolutional networks, questioning the necessity of different components in the pipeline. We find that max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks. Following this finding -- and building on other recent work for finding simple network structures -- we propose a new architecture that consists solely of convolutional layers and yields competitive or state of the art performance on several object recognition datasets (CIFAR-10, CIFAR-100, ImageNet). To analyze the network we introduce a new variant of the "deconvolution approach" for visualizing features learned by CNNs, which can be applied to a broader range of network structures than existing approaches.
This paper has not been read by Pith yet.
Forward citations
Cited by 18 Pith papers
-
Dataset Distillation
Dataset distillation creates a tiny synthetic training set that, when used with a fixed network initialization, produces models whose performance approximates that of models trained on the full original dataset.
-
Federated Learning: Strategies for Improving Communication Efficiency
Structured updates (low-rank or masked) and sketched updates (quantized, rotated, subsampled) reduce uplink communication in federated learning by up to two orders of magnitude on convolutional and recurrent networks.
-
Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
DCGANs with architectural constraints learn a hierarchy of representations from object parts to scenes in both generator and discriminator across image datasets.
-
Spectral Integrated Gradients for Coarse-to-Fine Feature Attribution
Spectral Integrated Gradients constructs SVD-based integration paths that activate singular components from largest to smallest, producing cleaner attribution maps and better quantitative scores than standard Integrat...
-
ExPath: Targeted Pathway Inference for Biological Knowledge Bases via Graph Learning and Explanation
ExPath is a subgraph inference framework that classifies bio-networks with experimental data and uses explanations to identify targeted pathways, reporting up to 4.5x higher Fidelity+ and 14x lower Fidelity- than base...
-
SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation
SalUn uses gradient-based weight saliency to achieve effective machine unlearning of data, classes, or concepts in image classification and generation, narrowing the gap to exact retraining.
-
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
Geometric deep learning provides a unified mathematical framework based on grids, groups, graphs, geodesics, and gauges to explain and extend neural network architectures by incorporating physical regularities.
-
Saliency-driven Word Alignment Interpretation for Neural Machine Translation
Saliency-driven interpretation methods reveal that NMT models learn word alignments of better quality than fast-align under force decoding and consistent with automatic tools under free decoding.
-
Seeing What Shouldn't Be There: Counterfactual GANs for Medical Image Attribution
A cycle-consistent GAN generates counterfactual medical images to attribute classification decisions more comprehensively than standard saliency methods.
-
Leveraging Convolutional Sparse Autoencoders for Robust Movement Classification from Low-Density sEMG
Convolutional sparse autoencoder on two-channel sEMG delivers 94.3% multi-subject F1 for six gestures, 92.3% after few-shot transfer to unseen subjects, and 90% after incremental extension to ten classes.
-
AdaProb: Efficient Machine Unlearning via Adaptive Probability
AdaProb performs machine unlearning by substituting final-layer output probabilities with optimized uniform pseudo-probabilities and updating model weights.
-
Explaining Graph Neural Networks for Node Similarity on Graphs
Empirical comparison shows gradient-based explanations for GNN node similarities are actionable, consistent, and retain effects when sparsified, unlike mutual information explanations.
-
Explaining the Explainers in Graph Neural Networks: a Comparative Study
Benchmark study of ten GNN explainers on eight architectures and six datasets that isolates usable components and issues practical recommendations.
-
Explaining an increase in predicted risk for clinical alerts
Methods are introduced to lift static attribution techniques to dynamical models for explaining risk increases in clinical alert systems.
-
Out-of-Distribution Detection Using Neural Rendering Generative Models
NRM enables OoD detection by joint latent likelihood, assigning lower values to SVHN than CIFAR-10 (unlike VAEs/flows) and consistent across other OoD sets.
-
Explainable Human Activity Recognition: A Unified Review of Concepts and Mechanisms
The paper delivers a mechanism-centric taxonomy and unified perspective on explainable human activity recognition methods across sensing modalities.
-
How Does Overparameterization Affect Machine Unlearning of Deep Neural Networks?
Overparameterized DNNs enable more effective machine unlearning for privacy and bias removal via localized decision-region adjustments, with performance depending on method access to forgotten data.
-
Image Classification with Hierarchical Multigraph Networks
Hierarchical multigraph GCNs applied to superpixels achieve competitive or superior accuracy to CNNs on standard image classification benchmarks.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.