REVIEW 4 cited by
Accurate INT8 Training Through Dynamic Block-Level Fallback
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Accurate INT8 Training Through Dynamic Block-Level Fallback
read the original abstract
Transformer models have achieved remarkable success across various AI applications but face significant training costs. Low-bit training, such as INT8 training, can leverage computational units with higher throughput, and has already demonstrated its effectiveness on GPT2 models with block-level quantization. However, it struggles with modern Transformer variants incorporating GLU units. This is because those variants demonstrate complex distributions of activation outliers. To address the challenge, we propose Fallback Quantization, implementing mixed-precision GEMM that dynamically falls back 8-bit to 16-bit for activation blocks containing outliers. Experiments show that our approach is robustly competent in both fine-tuning and pretraining settings. Moreover, our method achieves a 1.57x end-to-end training speedup on RTX4090 GPUs.
Forward citations
Cited by 4 Pith papers
-
HiFloat4 Format for End-To-End Reinforcement Learning Post-Training of Large Language Models
In end-to-end 4-bit RL post-training, rollout activation underflow, not training quantization, is the main accuracy killer; a sparse residual correction closes most of the gap to BF16.
-
Decomposing MXFP4 quantization error for LLM reinforcement learning: reducible bias, recoverable deadzone, and an irreducible floor
MXFP4 quantization error decomposes into scale bias, deadzone truncation, and grid noise; macro-block scaling, outlier fallback, and adaptive quantization noise recover BF16 accuracy to within 0.7% and 3.0% on tested models.
-
Decomposing MXFP4 quantization error for LLM reinforcement learning: reducible bias, recoverable deadzone, and an irreducible floor
MXFP4 error decomposes into scale bias, deadzone truncation, and grid noise that each dominate distinct RL failure modes, with macro-block scaling, outlier fallback, and adaptive noise recovering or exceeding BF16 per...
-
Decomposing MXFP4 quantization error for LLM reinforcement learning: reducible bias, recoverable deadzone, and an irreducible floor
MXFP4 quantization error decomposes into scale bias, deadzone truncation, and grid noise; mode-targeted corrections recover BF16 accuracy within 0.7% on Qwen2.5-3B and exceed it by 1.0% on Qwen3-30B-A3B.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.