Pith. sign in

REVIEW 3 cited by

Accurate INT8 Training Through Dynamic Block-Level Fallback

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2503.08040 v3 pith:Z5RRMBIO submitted 2025-03-11 cs.LG

Pengle Zhang , Jia Wei , Jintao Zhang , Jun Zhu , Jianfei Chen This is my paper · ORCID
classification cs.LG
keywords trainingactivationblock-levelfallbackint8modelsoutliersquantization
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Transformer models have achieved remarkable success across various AI applications but face significant training costs. Low-bit training, such as INT8 training, can leverage computational units with higher throughput, and has already demonstrated its effectiveness on GPT2 models with block-level quantization. However, it struggles with modern Transformer variants incorporating GLU units. This is because those variants demonstrate complex distributions of activation outliers. To address the challenge, we propose Fallback Quantization, implementing mixed-precision GEMM that dynamically falls back 8-bit to 16-bit for activation blocks containing outliers. Experiments show that our approach is robustly competent in both fine-tuning and pretraining settings. Moreover, our method achieves a 1.57x end-to-end training speedup on RTX4090 GPUs.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. HiFloat4 Format for End-To-End Reinforcement Learning Post-Training of Large Language Models

    cs.LG 2026-07 conditional novelty 7.0 of 10

    In end-to-end 4-bit RL post-training, rollout activation underflow, not training quantization, is the main accuracy killer; a sparse residual correction closes most of the gap to BF16.

  2. Scaling Law for Quantization-Aware Training

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A unified QAT scaling law predicts 4-bit quantization error from model size, training tokens, and group size, showing activation outliers in the FC2 layer are the main W4A4 bottleneck.

  3. PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Models

    cs.CV 2025-06 conditional novelty 4.0 of 10

    PAROAttention permutes tokens along frame, height, and width axes to make visual attention block-wise, enabling sparse and INT8/INT4 quantized attention with near-baseline generation quality.

Pith tools