Pith. sign in

REVIEW 3 cited by

NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.20650 v1 pith:BFDCNHZG submitted 2024-10-28 cs.LG cs.AIstat.ML

Yongchang Hao , Yanshuai Cao , Lili Mou This is my paper · ORCID
classification cs.LGcs.AIstat.ML
keywords traininginferenceperformancememorynetworksneuralneuzipavailable
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The performance of neural networks improves when more parameters are used. However, the model sizes are constrained by the available on-device memory during training and inference. Although applying techniques like quantization can alleviate the constraint, they suffer from performance degradation. In this work, we introduce NeuZip, a new weight compression scheme based on the entropy of floating-point numbers in neural networks. With NeuZip, we are able to achieve memory-efficient training and inference without sacrificing performance. Notably, we significantly reduce the memory footprint of training a Llama-3 8B model from 31GB to less than 16GB, while keeping the training dynamics fully unchanged. In inference, our method can reduce memory usage by more than half while maintaining near-lossless performance. Our code is publicly available.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Lossless Tensor Compression as Program Synthesis

    cs.SE 2026-08 conditional novelty 7.0 of 10

    By expressing each tensor as a synthesized reversible program and storing the shortest one, Brevis losslessly compresses 2.13 TB of model checkpoints to 1.41 TB, beating ZipNN, zstd, gzip, LZ4, and Snappy.

  2. Huff-LLM: End-to-End Lossless Compression for Efficient LLM Inference

    cs.LG 2025-02 conditional novelty 6.0 of 10

    Huff-LLM splits FP16/BF16 LLM weights into small bit groups, Huffman-compresses each group, and uses custom hardware decoders so weights stay compressed through the memory hierarchy during inference.

  3. Scalable Parameter and Memory Efficient Pretraining for LLM: Recent Algorithmic Advances and Benchmarking

    cs.LG 2025-05 conditional novelty 4.0 of 10

    A benchmark and two low-cost tricks (weight refactorization and momentum reset) that make low-rank LLM pre-training competitive with GaLore and Fira at about 25% lower memory.

Pith tools