AlphaQ performs calibration-free mixed-precision quantization of MoE models by allocating higher bits to experts whose weight spectra exhibit stronger heavy-tailed structure according to HT-SR theory, outperforming calibration-based methods and reaching near full-precision accuracy at 3.5 average bi
Mahoney, and Kurt Keutzer
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
years
2026 3representative citing papers
Switchless topologies such as 3D full-mesh are 20.6-56.2% more cost-effective than scale-up networks for MoE LLM serving, with current link bandwidths over-provisioned by up to 27%.
By restricting token mixing to rank-1 decayed matrices, the same model trains in parallel but generates recurrently with constant memory, yielding large throughput gains in compute-matched benchmarks.
citing papers explorer
-
AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization
AlphaQ performs calibration-free mixed-precision quantization of MoE models by allocating higher bits to experts whose weight spectra exhibit stronger heavy-tailed structure according to HT-SR theory, outperforming calibration-based methods and reaching near full-precision accuracy at 3.5 average bi
-
Rethinking Network Topologies for Cost-Effective Mixture-of-Experts LLM Serving
Switchless topologies such as 3D full-mesh are 20.6-56.2% more cost-effective than scale-up networks for MoE LLM serving, with current link bandwidths over-provisioned by up to 27%.
-
Structured Recurrent Mixers for Massively Parallelized Sequence Generation
By restricting token mixing to rank-1 decayed matrices, the same model trains in parallel but generates recurrently with constant memory, yielding large throughput gains in compute-matched benchmarks.