REVIEW 8 cited by
Scalable MatMul-free Language Modeling
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Large Language Models (LLMs) have fundamentally altered how we approach scaling in machine learning. However, these models pose substantial computational and memory challenges, primarily due to the reliance on matrix multiplication (MatMul) within their attention and feed-forward (FFN) layers. We demonstrate that MatMul operations can be eliminated from LLMs while maintaining strong performance, even at billion-parameter scales. Our MatMul-free models, tested on models up to 2.7B parameters, are comparable to state-of-the-art pre-trained Transformers, and the performance gap narrows as model size increases. Our approach yields significant memory savings: a GPU-efficient implementation reduces memory consumption by up to 61% during training and over 10x during inference. When adapted for a multi-chip neuromorphic system, the model leverages asynchronous processing to achieve 4x higher throughput with 10x less energy than edge GPUs.
Forward citations
Cited by 8 Pith papers
-
All-optical temporal integration mediated by subwavelength heat antennas
A ring resonator with titanium nano-heaters absorbs control light at standing-wave antinodes, heats up, and shifts a probe wavelength, performing all-optical temporal integration and nonlinear activation in a single device.
-
PROM: Prioritize Reduction of Multiplications Over Lower Bit-Widths for Efficient CNNs
PROM quantizes pointwise convolutions to ternary weights and all other layers to 8 bits, converting the dominant CNN operations to additions and yielding order-of-magnitude estimated energy savings at similar accuracy.
-
Neuromorphic Principles for Efficient Large Language Models on Intel Loihi 2
A 370M MatMul-free LLM is mapped onto Intel Loihi 2 and reported to achieve up to 3x higher generation throughput with about 2x less energy than transformer LLMs on an edge GPU, based on preliminary measurements that ...
-
RandLoRA: Full-rank parameter-efficient fine-tuning of large models
RandLoRA achieves full-rank weight updates in parameter-efficient fine-tuning by learning diagonal scalings over fixed random low-rank bases, outperforming LoRA across vision and language tasks.
-
Learnable Sparsification of Die-to-Die Communication via Spike-Based Encoding
A hybrid accelerator that confines spiking layers to die-to-die interfaces offers simulated latency and energy gains over all-ANN designs while matching accuracy.
-
Threshold Neuron: A Brain-inspired Artificial Neuron for Efficient On-device Inference
The paper presents a multiplication-free threshold neuron that builds unified networks with large claimed hardware savings, but the fairest comparisons are still missing.
-
Sparse Mutual Information Graph Averaging for Improving Random Indexing Embeddings
On 272 family-category analogy questions from a fairytales corpus, averaging random-indexing vectors over a top-50 PPMI graph raised accuracy from 19.4% to 30.7% across five seeds, while the same averaging hurt strong...
-
An Extra RMSNorm is All You Need for Fine Tuning to 1.58 Bits
Fine-tuning a pretrained LLM with extra RMSNorm before each quantized linear layer and a gradual two-phase quantization schedule can reach ternary 1.58-bit weights, according to the paper, but the supporting evidence ...
Discussion (0). Continue with ORCID to comment.