RECORD
AlignQuant: Tile-Aligned Mixed-Precision Quantization for Efficient LLM Generation
Machine Learning · Submitted 5 Oct 2026
abstract compiled · 2026-10-07 · no language-model review
The authors' main claim These results show that local precision flexibility and regular GPU execution can coexist through a shared tile unit.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Fine-grained mixed-precision quantization promises efficient large language model inference, but local precision choices can conflict with regular GPU storage and computation units. This precision-boundary mismatch limits the translation of compression into practical acceleration. We introduce AlignQuant, a post-training quantization method that uses GPU-compatible two-dimensional weight tiles as the common unit of precision allocation, compact storage, and execution. This shared partition lets precision follow sensitivity within output channels. Joint prefill/decode calibration scores precision reductions using projection-output perturbations weighted by language-model loss gradients under quantized activations. Phase-normalized scores prioritize higher precision for tiles important to either phase under a model-wide weight-storage budget. Each tile stores one selected representation, while phase-specialized kernels reuse the packed model and expand lower-bit weights for INT8 computation with 8-bit activations. Across four LLMs spanning 3B to 14B parameters, AlignQuant achieves up to $2.50\times$ generation speedup over BF16 while preserving model quality. Evaluations further cover three GPUs and contexts up to 64K tokens. These results show that local precision flexibility and regular GPU execution can coexist through a shared tile unit. The implementation is available at https://github.com/HanzhiZhang-Ulrica/AlignQuant.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Fine-grained mixed-precision quantization promises efficient large language model inference, but local precision choices can conflict with regular GPU storage and computation units. | background 0.96 | 0.07 | 0.03 | asserted 0.70 |
| s1 | This precision-boundary mismatch limits the translation of compression into practical acceleration. | background 0.73 | 0.09 | 0.03 | asserted 1.00 |
| s2 | We introduce AlignQuant, a post-training quantization method that uses GPU-compatible two-dimensional weight tiles as the common unit of precision allocation, compact storage, and execution. | contribution 0.92 | 0.90 | 0.12 | asserted 1.00 |
| s3 | This shared partition lets precision follow sensitivity within output channels. | method 0.82 | 0.55 | 0.08 | asserted 0.99 |
| s4 | Joint prefill/decode calibration scores precision reductions using projection-output perturbations weighted by language-model loss gradients under quantized activations. | method 0.99 | 0.92 | 0.04 | asserted 0.98 |
| s5 | Phase-normalized scores prioritize higher precision for tiles important to either phase under a model-wide weight-storage budget. | method 0.99 | 0.83 | 0.06 | asserted 0.99 |
| s6 | Each tile stores one selected representation, while phase-specialized kernels reuse the packed model and expand lower-bit weights for INT8 computation with 8-bit activations. | method 1.00 | 0.79 | 0.06 | asserted 1.00 |
| s7 | Across four LLMs spanning 3B to 14B parameters, AlignQuant achieves up to $2.50\times$ generation speedup over BF16 while preserving model quality. | result 1.00 | 0.97 | 0.41 | asserted 1.00 |
| s8 | Evaluations further cover three GPUs and contexts up to 64K tokens. | result 0.45 | 0.93 | 0.03 | asserted 0.95 |
| s9 | These results show that local precision flexibility and regular GPU execution can coexist through a shared tile unit. | interpretation 1.00 | 0.85 | 0.85 | asserted 1.00 |
| s10 | The implementation is available at https://github.com/HanzhiZhang-Ulrica/AlignQuant. | contribution 0.67 | 0.47 | 0.02 | asserted 0.98 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Fine-grained mixed-precision quantization promises efficient large language model inference, but local precision choices can conflict with regular GPU storage and computation units. This precision-boundary mismatch limits the translation of compression into practical acceleration. We introduce AlignQuant, a post-training quantization method that uses GPU-compatible two-dimensional weight tiles as the common unit of precision allocation, compact storage, and execution. This shared partition lets precision follow sensitivity within output channels. Joint prefill/decode calibration scores precision reductions using projection-output perturbations weighted by language-model loss gradients under quantized activations. Phase-normalized scores prioritize higher precision for tiles important to either phase under a model-wide weight-storage budget. Each tile stores one selected representation, while phase-specialized kernels reuse the packed model and expand lower-bit weights for INT8 computation with 8-bit activations. Across four LLMs spanning 3B to 14B parameters, AlignQuant achieves up to $2.50\times$ generation speedup over BF16 while preserving model quality. Evaluations further cover three GPUs and contexts up to 64K tokens. These results show that local precision flexibility and regular GPU execution can coexist through a shared tile unit. The implementation is available at https://github.com/HanzhiZhang-Ulrica/AlignQuant.
Human review
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.