AGoQ delivers up to 52% lower memory use and 1.34x faster training for 8B-32B LLaMA models by using near-4-bit adaptive activations and 8-bit gradients while preserving pretraining convergence and downstream accuracy.
arXiv preprint arXiv:2508.00806 , year=
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
years
2026 2verdicts
UNVERDICTED 2representative citing papers
Presents quantization, checkpointing, softmax approximation, and logits masking to achieve substantial peak memory reductions in LoRA fine-tuning of 3B LLMs.
citing papers explorer
-
AGoQ: Activation and Gradient Quantization for Memory-Efficient Distributed Training of LLMs
AGoQ delivers up to 52% lower memory use and 1.34x faster training for 8B-32B LLaMA models by using near-4-bit adaptive activations and 8-bit gradients while preserving pretraining convergence and downstream accuracy.
-
Techniques for Peak Memory Reduction for LoRA Fine-tuning of LLMs on Edge Devices
Presents quantization, checkpointing, softmax approximation, and logits masking to achieve substantial peak memory reductions in LoRA fine-tuning of 3B LLMs.