StoMPP progressively binarizes BNN layers layerwise from input to output via stochastic masks, delivering depth-scalable accuracy gains in a fully STE-free regime by controlling activation-induced gradient blockades.
hub
Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
22 Pith papers cite this work. Polarity classification is still indexing.
22
Pith papers citing it
abstract
We introduce a method to train Binarized Neural Networks (BNNs) - neural networks with binary weights and activations at run-time. At training-time the binary weights and activations are used for computing the parameters gradients. During the forward pass, BNNs drastically reduce memory size and accesses, and replace most arithmetic operations with bit-wise operations, which is expected to substantially improve power-efficiency. To validate the effectiveness of BNNs we conduct two sets of experiments on the Torch7 and Theano frameworks. On both, BNNs achieved nearly state-of-the-art results over the MNIST, CIFAR-10 and SVHN datasets. Last but not least, we wrote a binary matrix multiplication GPU kernel with which it is possible to run our MNIST BNN 7 times faster than with an unoptimized GPU kernel, without suffering any loss in classification accuracy. The code for training and running our BNNs is available on-line.
hub tools
citation-role summary
method 2
background 1
citation-polarity summary
representative citing papers
FTerViT introduces fully ternary Vision Transformers with TernaryBitConv2d and TernaryLayerNorm operators, achieving 82.43% ImageNet top-1 at 6.09 MB with 15x compression.
Proposes first stochastic-computing DNN acceleration framework tailored to AQFP superconducting technology.
LLM.int8() performs 8-bit inference for transformers up to 175B parameters with no accuracy loss by combining vector-wise quantization for most features with 16-bit mixed-precision handling of systematic outlier dimensions.
A crossing activation function combined with virtual noise fields allows one neural network to learn multiple functions assigned to different noise locations, with capacity rising when noise arrangement matches function proximity.
LoRDBA replaces LoRA low-rank factors with binary sign carriers and channel-wise scales, matching fp16 LoRA quality in some regimes with over 10x smaller adapter size and at most 8% prefill latency overhead.
Introduces the first heterogeneous multi-source mmWave point cloud HAR dataset and DAP-Net, which uses Doppler patterns for source-invariant action recognition and outperforms prior methods.
SURGE proposes a dual-path gradient compensator and adaptive gradient scaler to mitigate gradient mismatch in binary neural network training via auxiliary backpropagation.
A progressive training scheme with binary-aware initialization and dual-scaling allows pre-trained LLMs to be converted to high-performance 1-bit models without training from scratch.
Proves polynomial-in-width and exponential-in-depth lower bounds on linear regions for ternary ReLU regression networks, with width-doubling constructions achieving bounds comparable to unrestricted ReLU networks.
A co-evolutionary compression technique reduces parameters and FLOPs in unpaired image-to-image translation GAN generators while maintaining translation quality on benchmarks.
Neural networks are trained as timing models of programs and analyzed via MILP to detect and quantify timing side-channel information leaks.
Radix-5 memristor crossbar CNN accelerator reaches 90.5% CIFAR-10 accuracy with 46% area reduction by using variable memristor counts and single-column signed weights.
LBLLM achieves better accuracy than prior binarization methods for LLMs by decoupling weight and activation quantization through initialization, layer-wise distillation, and learnable activation scaling.
A domain-specific reduced RISC-V core for Tsetlin Machine inference delivers up to 98% faster execution and 29.7x lower energy than baseline RV32IM while matching or exceeding BNN accuracy on tested datasets.
QuoVLA introduces a quotient-space framework that compresses VLM latents into action-sufficient representations via quantization and dual-branch design for better VLA generalization.
Replacing pointwise convolutions with DWHT yields a model with 79.1% fewer parameters, 48.4% fewer FLOPs, and 1.49% higher accuracy than MobileNet-V1 on CIFAR-100.
CNNs applied to global history improve prediction accuracy for hard-to-predict branches in SPEC 2017, with hardware-adapted inference and reusability across inputs.
HTAF is a sigmoid-tanh composite that approximates the Heaviside function to allow stable gradient training of binary activation networks, yielding ICBMs with stable discretization and competitive performance on image tasks.
A BNN-based YOLOv3-tiny-like object detector with 1-bit weights and 8-bit activations is implemented in Verilog on FPGA, achieving 39.6% mAP50 on VOC and 0.999964 correlation with the ONNX model in RTL simulation.
Hybrid method applies pruning and quantization followed by MoE routing of compressed CNN experts to achieve large reductions in FLOPs and parameters with negligible accuracy loss on benchmarks.
A comprehensive review of deep learning techniques for computational mechanics, including LSTM for constitutive modeling, PINNs for PDE solving, optimizers, and kernel methods.
citing papers explorer
-
Layerwise Progressive Freezing: A Training Scaffold for Depth-Scalable Binary Networks
StoMPP progressively binarizes BNN layers layerwise from input to output via stochastic masks, delivering depth-scalable accuracy gains in a fully STE-free regime by controlling activation-induced gradient blockades.
-
FTerViT: Fully Ternary Vision Transformer
FTerViT introduces fully ternary Vision Transformers with TernaryBitConv2d and TernaryLayerNorm operators, achieving 82.43% ImageNet top-1 at 6.09 MB with 15x compression.
-
A Stochastic-Computing based Deep Learning Framework using Adiabatic Quantum-Flux-Parametron SuperconductingTechnology
Proposes first stochastic-computing DNN acceleration framework tailored to AQFP superconducting technology.
-
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
LLM.int8() performs 8-bit inference for transformers up to 175B parameters with no accuracy loss by combining vector-wise quantization for most features with 16-bit mixed-precision handling of systematic outlier dimensions.
-
Spatial Partial Functionalization of Neural Networks based on Noise Fields
A crossing activation function combined with virtual noise fields allows one neural network to learn multiple functions assigned to different noise locations, with capacity rising when noise arrangement matches function proximity.
-
Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuning
LoRDBA replaces LoRA low-rank factors with binary sign carriers and channel-wise scales, matching fp16 LoRA quality in some regimes with over 10x smaller adapter size and at most 8% prefill latency overhead.
-
DAP: Doppler-aware Point Network for Heterogeneous mmWave Action Recognition
Introduces the first heterogeneous multi-source mmWave point cloud HAR dataset and DAP-Net, which uses Doppler patterns for source-invariant action recognition and outperforms prior methods.
-
SURGE: Surrogate Gradient Adaptation in Binary Neural Networks
SURGE proposes a dual-path gradient compensator and adaptive gradient scaler to mitigate gradient mismatch in binary neural network training via auxiliary backpropagation.
-
Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Models
A progressive training scheme with binary-aware initialization and dual-scaling allows pre-trained LLMs to be converted to high-performance 1-bit models without training from scratch.
-
A Lower Bound for the Number of Linear Regions of Ternary ReLU Regression Neural Networks
Proves polynomial-in-width and exponential-in-depth lower bounds on linear regions for ternary ReLU regression networks, with width-doubling constructions achieving bounds comparable to unrestricted ReLU networks.
-
Co-Evolutionary Compression for Unpaired Image Translation
A co-evolutionary compression technique reduces parameters and FLOPs in unpaired image-to-image translation GAN generators while maintaining translation quality on benchmarks.
-
Efficient Detection and Quantification of Timing Leaks with Neural Networks
Neural networks are trained as timing models of programs and analyzed via MILP to detect and quantify timing side-channel information leaks.
-
Adaptive Precision CNN Accelerator Using Radix-X Parallel Connected Memristor Crossbars
Radix-5 memristor crossbar CNN accelerator reaches 90.5% CIFAR-10 accuracy with 46% area reduction by using variable memristor counts and single-column signed weights.
-
LBLLM: Lightweight Binarization of Large Language Models via Three-Stage Distillation
LBLLM achieves better accuracy than prior binarization methods for LLMs by decoupling weight and activation quantization through initialization, layer-wise distillation, and learnable activation scaling.
-
Low-Energy Reduced RISC-V Instruction Subset Processor for Tsetlin Machine Inference at the Edge
A domain-specific reduced RISC-V core for Tsetlin Machine inference delivers up to 98% faster execution and 29.7x lower energy than baseline RV32IM while matching or exceeding BNN accuracy on tested datasets.
-
QuoVLA: Quotient Space for Vision-Language-Action Models
QuoVLA introduces a quotient-space framework that compresses VLM latents into action-sufficient representations via quantization and dual-branch design for better VLA generalization.
-
New pointwise convolution in Deep Neural Networks through Extremely Fast and Non Parametric Transforms
Replacing pointwise convolutions with DWHT yields a model with 79.1% fewer parameters, 48.4% fewer FLOPs, and 1.49% higher accuracy than MobileNet-V1 on CIFAR-100.
-
Improving Branch Prediction By Modeling Global History with Convolutional Neural Networks
CNNs applied to global history improve prediction accuracy for hard-to-predict branches in SPEC 2017, with hardware-adapted inference and reusability across inputs.
-
A Composite Activation Function for Learning Stable Binary Representations
HTAF is a sigmoid-tanh composite that approximates the Heaviside function to allow stable gradient training of binary activation networks, yielding ICBMs with stable discretization and competitive performance on image tasks.
-
Design and Implementation of BNN-Based Object Detection on FPGA
A BNN-based YOLOv3-tiny-like object detector with 1-bit weights and 8-bit activations is implemented in Verilog on FPGA, achieving 39.6% mAP50 on VOC and 0.999964 correlation with the ONNX model in RTL simulation.
-
Hybrid Compression: Integrating Pruning and Quantization for Optimized Neural Networks
Hybrid method applies pruning and quantization followed by MoE routing of compressed CNN experts to achieve large reductions in FLOPs and parameters with negligible accuracy loss on benchmarks.
-
Deep learning applied to computational mechanics: A comprehensive review, state of the art, and the classics
A comprehensive review of deep learning techniques for computational mechanics, including LSTM for constitutive modeling, PINNs for PDE solving, optimizers, and kernel methods.