REVIEW 2 major objections 4 minor 61 references
FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling
T0 review · 2 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read The paper claims that the quantization scale in FP4 LLM quantization can be fully decoupled from the hardware-mandated dequantization scale, and that learning this free scale in full precision recovers nearly all FP16 accuracy without any i
desk verdict A genuinely useful scale-decoupling idea with mostly clean experiments, but a WikiText2-below-FP16 anomaly and a missing appendix mean the SOTA claim needs scrutiny before I'd fully trust it. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The key object is the pair (S^q, S^{dq})—the quantization and dequantization scales. Standard methods enforce S^q = S^{dq} = Q_fmt(S^{fp}), where Q_fmt is the low-precision scale format (E8M0 for MXFP4, E4M3 for NVFP4). FOCUS relaxes this to S^q = S^{dq} * c, with c a learnable full-precision coefficient (Coupled-Relaxation Scaling), and further splits each hardware block into sub-blocks that each get their own coefficient (Dual-Granularity Scaling). This lifts scale optimization from a discrete, hardware-constrained space to a continuous, gradient-optimizable space while keeping the deployed S^{dq} fully compliant. The learning loop uses straight-through estimation for non-differentiable qu
What would settle it
Calibrate FOCUS on WikiText2 and evaluate on a held-out corpus distinct from WikiText2 and C4 (e.g., a code dataset or a different-domain benchmark); if the perplexity or accuracy advantage over a hardware-constrained baseline like MR-GPTQ disappears or reverses, the free-scale optimization is overfitting to the calibration distribution rather than finding genuinely better FP4 assignments.
Extended reading notes
Core claim
The central discovery is that the two scales in FP4 quantization serve different roles and need not be equal: the dequantization scale is stored and must conform to the hardware format and block size, but the quantization scale is only used offline to assign FP4 values and is never retained. FOCUS exploits this asymmetry by expressing the quantization scale as the dequantization scale times a learnable full-precision coefficient (Coupled-Relaxation Scaling), and by letting that coefficient vary per sub-block within a hardware block (Dual-Granularity Scaling). Because the coefficients are discarded after calibration, the deployed model remains standard MXFP4/NVFP4 format with no additional in
Load-bearing premise
The method's gains rest on the assumption that the 1,248-sample WikiText2 calibration set, with a sequence length of 2,048, is representative enough that the learned scale coefficients transfer to other text distributions, zero-shot reasoning tasks, and multi-step math reasoning.
Editorial extensions
If this is right
- FP4 quantization can be improved through gradient-based scale learning without adding any runtime cost, since only the standard FP4 weights and dequantization scales are stored.
- MXFP4, whose E8M0 scales are inherently coarse, benefits enough to approach FP16 language modeling perplexity, making the cheaper format more viable for deployment.
- The relaxation trick is complementary to transform-based methods (e.g., Hadamard rotations), so future pipelines can combine offline scale relaxation with online transforms for further accuracy.
- Quantization time remains practical for post-training use—about 19 minutes on a single GPU for a 4-billion-parameter model—making the method a drop-in replacement for existing scale-search steps.
Reading between the lines
- If the quantization scale is truly unconstrained, the same decoupling idea could apply to other low-precision formats such as FP8 or MXFP6, where dequantization scales are also format-limited, potentially yielding similar accuracy gains.
- The reported WikiText2 perplexity for Qwen3-4B MXFP4 is lower than the FP16 baseline, which hints that the learned scales may be overfit to the calibration distribution; a different calibration corpus or evaluation split would test whether the gains generalize.
- The sub-block coefficients effectively act as per-sub-block rounding-boundary shifts; this suggests a broader design space where offline quantization parameters can be tuned for purposes other than minimizing reconstruction error, such as preserving downstream task logits.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FOCUS, a post-training quantization method for FP4 formats (MXFP4 and NVFP4). Its central observation is that the quantization scale used offline to map weights to FP4 is never stored and therefore need not satisfy the hardware precision/block-size constraints imposed on the dequantization scale. FOCUS implements this via Coupled-Relaxation Scaling (CRS), a learnable full-precision coefficient multiplying the hardware-compliant dequantization scale, and Dual-Granularity Scaling (DGS), which applies independent sub-block coefficients during quantization while leaving the deployed format unchanged. Experiments on Qwen3 and LLaMA-3.1/3.2 models report state-of-the-art FP4 perplexity and zero-shot accuracy under both formats, with a concrete cost comparison and no inference-time overhead.
Significance. If the empirical claims survive scrutiny, FOCUS is a conceptually clean and practically useful contribution: it exploits a genuinely unused degree of freedom (the offline-only quantization scale), preserves hardware compatibility, and is orthogonal to transform-based methods. The ablations cleanly separate CRS and DGS, the quantization-cost table is concrete, and the authors commit to releasing code. The main risk is that the learned scale/coefficient optimization is heavily overparameterized relative to the calibration data; Table 1 shows a calibration-specialization signature (WikiText2 perplexity below FP16 while C4 remains above FP16). The missing appendix also prevents verification of several claims that motivate the design and the comparison. These are addressable, so the central idea is defensible pending stronger generalization evidence.
major comments (2)
- [Experiments, Tables 1 and 4] The Qwen3-4B results show FOCUS below the FP16 reference on WikiText2 (12.85 MXFP4, 12.57 NVFP4 vs 13.66 FP16) while remaining above FP16 on C4 (17.77/16.97 vs 16.63). Table 4 shows even the naive scale-learning baseline already reaches 12.96 on Wiki2. Since calibration uses 1,248 WikiText2 training sequences (~2.5M tokens) and the optimized parameters number roughly 0.16-0.19 per element (~6e8 for a 4B model), this pattern is consistent with specialization to the calibration distribution rather than a generally better FP4 representation. The SOTA claim spans C4, zero-shot, and reasoning benchmarks, so the paper should demonstrate cross-corpus generalization: calibrate on C4 (or a second corpus), evaluate on multiple held-out sets, and report calibration-seed/split variance. Without this, the headline numbers do not establish a general accuracy improvement.
- [Methodology/CRS and 'FOCUS without Extra Transforms'; missing Appendix] Several load-bearing results are deferred to an appendix that is not present in the submitted text: the claimed full-decoupling training instability that motivates CRS; the Hadamard fusion experiments and re-quantization degradation; the negligible additional training cost; and the claimed complementarity/combinations with transform-based methods. The CRS design rationale and the comparison to transform-based SOTA cannot be checked without these results. The authors should either provide the appendix in full or remove/qualify the claims that depend on it.
minor comments (4)
- [Experimental Settings] The KL-TopLoss objective with k=1000 is referenced to Hu et al. but not defined; please specify the exact loss (e.g., top-k token KL divergence) and how k affects optimization.
- [Table 3] GSM8K/MMLU CoT results are reported only for MXFP4, although the abstract claims accuracy under both formats. Please add NVFP4 reasoning results or qualify the claim.
- [Related Work] Related Work credits SOAR (Bao et al. 2026) with 'decoupled scale search'; the introduction's statement that existing methods 'tightly couple' quantization and dequantization scales should be qualified to avoid an apparent contradiction.
- [Tables and text] There are formatting/spacing issues (e.g., '12.8517.77' in Table 5 and missing spaces in phrases such as 'FOCUSiscomplementary' and 'Sq i' constructions). Please proofread the final version.
Circularity Check
No significant circularity: FOCUS's central claims are supported by independent held-out evaluation; the overlapping-author SOAR citation is not load-bearing.
full rationale
The paper's derivation chain does not reduce to its inputs. The key premise that the quantization scale is free from hardware constraints follows directly from the deployment description in Eqs. (5)-(7): Sq is used only offline and discarded, while Sdq is stored, so only Sdq must conform to E4M3/E8M0. This is an observation about the method's own forward pass, not a conclusion imported from a self-citation. CRS and DGS are validated by training the relaxation coefficients on a WikiText2 calibration subset and measuring generalization on held-out C4, zero-shot tasks, GSM8K, and MMLU; no fitted scale or coefficient is relabeled as a prediction. The overlapping-author citation (Bao et al. 2026, SOAR) appears only as related-work context and is not used to justify FOCUS's claims or to rule out alternatives; there is no imported uniqueness theorem and no ansatz smuggled in via citation. Several details are deferred to a missing appendix (full-decoupling instability, Hadamard-fusion experiments, transform combination), which is a verifiability/completeness concern rather than a circularity concern. Overall, the empirical results stand as external evidence, so the circularity score is 0.
Assumptions & free parameters
free parameters (6)
- Relaxation coefficient c_i (CRS) =
learned, init=1
- Sub-block coefficients c^k_i (DGS) =
learned, init=1
- Scale learning rates =
MXFP4: 2e-2 (scale), 5e-2 (coef); NVFP4: 5e-3 (scale), 1e-3 (coef)
- Sub-block size =
8 for both formats
- KL-TopLoss k =
1000
- Calibration samples =
1248 WikiText2 training sequences
assumptions (5)
- domain assumption Straight-Through Estimator provides usable gradients through non-differentiable quantization
- domain assumption FP4 formats (MXFP4 and NVFP4) are correctly represented by the block-scaling equations in Eq. 1-3
- domain assumption The quantization scale is never stored at inference
- domain assumption Knowledge distillation loss (KL-TopLoss) is an effective surrogate for task-level accuracy
- domain assumption Calibration data (WikiText2 training) is representative of evaluation distributions
Cite this review
Pith. "Pith review of FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling." pith.science (2026). https://pith.science/paper/QMGIOGXW
@misc{pith2026260801847,
author = {Pith},
title = {Pith review of: FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling},
year = {2026},
howpublished = {\url{https://pith.science/paper/QMGIOGXW}},
note = {Machine review of arXiv:2608.01847}
}
read the original abstract
Large language models (LLMs) achieve remarkable performance but are expensive to deploy due to their enormous size. FP4 quantization, with formats such as MXFP4 and NVFP4, offers an appealing solution with native hardware support on modern accelerators. However, maintaining accuracy under FP4 precision remains difficult. A key bottleneck lies in scale optimization: existing methods tightly couple the quantization and dequantization scales, forcing both to conform to the discrete low-precision format required by hardware, such as E8M0 in MXFP4. Yet the quantization scale is never stored and need not obey this constraint, suggesting a significant untapped optimization space. In this work, we propose FOCUS, a post-training quantization framework with end-to-end scale learning for FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling. Coupled-Relaxation Scaling (CRS) relaxes the tight coupling between quantization and dequantization scales with a learnable full-precision coefficient, enabling more effective optimization without breaking hardware compliance. Dual-Granularity Scaling (DGS) further refines the quantization scale at a finer sub-block granularity, allowing more precise adaptation to local weight distributions. Experiments across multiple LLM families and benchmarks show that FOCUS achieves state-of-the-art FP4 accuracy under both MXFP4 and NVFP4 formats, while introducing no additional inference overhead. Code and quantized models will be released at https://github.com/tencent/AngelSlim.
Figures
Reference graph
Works this paper leans on
-
[1]
Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education
Clancey, William J. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83)
-
[2]
Classification Problem Solving
Clancey, William J. Classification Problem Solving. Proceedings of the Fourth National Conference on Artificial Intelligence
-
[3]
, title =
Robinson, Arthur L. , title =. 1980 , doi =. https://science.sciencemag.org/content/208/4447/1019.full.pdf , journal =
1980
-
[4]
New Ways to Make Microcircuits Smaller---Duplicate Entry
Robinson, Arthur L. New Ways to Make Microcircuits Smaller---Duplicate Entry. Science
-
[5]
Clancey and Glenn Rennels , abstract =
Diane Warner Hasling and William J. Clancey and Glenn Rennels , abstract =. Strategic explanations for a diagnostic consultation system , journal =. 1984 , issn =. doi:https://doi.org/10.1016/S0020-7373(84)80003-6 , url =
-
[6]
and Rennels, Glenn R
Hasling, Diane Warner and Clancey, William J. and Rennels, Glenn R. and Test, Thomas. Strategic Explanations in Consultation---Duplicate. The International Journal of Man-Machine Studies
-
[7]
Poligon: A System for Parallel Problem Solving
Rice, James. Poligon: A System for Parallel Problem Solving
-
[8]
Transfer of Rule-Based Expertise through a Tutorial Dialogue
Clancey, William J. Transfer of Rule-Based Expertise through a Tutorial Dialogue
Show all 61 references
-
[9]
The Engineering of Qualitative Models
Clancey, William J. The Engineering of Qualitative Models
-
[10]
2023 , eprint=
Attention Is All You Need , author=. 2023 , eprint=
2023
-
[11]
Pluto: The 'Other' Red Planet
NASA. Pluto: The 'Other' Red Planet
-
[12]
arXiv preprint arXiv:2512.02010 , year=
Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling , author=. arXiv preprint arXiv:2512.02010 , year=
-
[13]
2023 , booktitle=
OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models , author=. 2023 , booktitle=
2023
-
[14]
LRQuant: Learnable and Robust Post-Training Quantization for Large Language Models , author=
-
[15]
SpinQuant: LLM Quantization with Learned Rotations , author=
-
[16]
AffineQuant: Affine Transformation Quantization for Large Language Models , author=
-
[17]
FlatQuant: Flatness Matters for LLM Quantization , author=
-
[18]
Up or Down? Adaptive Rounding for Post-Training Quantization , author=
-
[19]
BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction , author=
-
[20]
OSTQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting , author=
-
[21]
Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization , author=
-
[22]
Block Rotation is All You Need for MXFP4 Quantization , author=
-
[23]
ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs , author=
-
[24]
Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction , author=
-
[25]
BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization , author=
-
[26]
arXiv preprint arXiv:2310.10537 , year=
Microscaling Data Formats for Deep Learning , author=. arXiv preprint arXiv:2310.10537 , year=
-
[27]
arXiv preprint arXiv:2509.25149 , year=
Pretraining Large Language Models with NVFP4 , author=. arXiv preprint arXiv:2509.25149 , year=
-
[28]
arXiv preprint arXiv:2601.09555 , year=
Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formats , author=. arXiv preprint arXiv:2601.09555 , year=
-
[29]
arXiv preprint arXiv:2604.17789 , year=
DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantization , author=. arXiv preprint arXiv:2604.17789 , year=
-
[30]
arXiv preprint arXiv:2603.22370 , year=
FAAR: Format-Aware Adaptive Rounding for NVFP4 , author=. arXiv preprint arXiv:2603.22370 , year=
-
[31]
Frantar, Elias and Ashkboos, Saleh and Hoefler, Torsten and Alistarh, Dan , booktitle =
-
[32]
Lin, Ji and Tang, Jiaming and Tang, Haotian and Yang, Shang and Chen, Wei-Ming and Wang, Wei-Chen and Xiao, Guangxuan and Dang, Xingyu and Gan, Chuang and Han, Song , booktitle=
-
[33]
arXiv preprint arXiv:2505.09388 , year =
Qwen3 Technical Report , author =. arXiv preprint arXiv:2505.09388 , year =
-
[34]
arXiv preprint arXiv:2407.21783 , year=
The llama 3 herd of models , author=. arXiv preprint arXiv:2407.21783 , year=
-
[35]
arXiv preprint arXiv:2501.12948 , year=
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=
-
[36]
arXiv preprint arXiv:2507.20534 , year=
Kimi k2: Open agentic intelligence , author=. arXiv preprint arXiv:2507.20534 , year=
-
[37]
arXiv preprint arXiv:2602.15763 , year=
Glm-5: from vibe coding to agentic engineering , author=. arXiv preprint arXiv:2602.15763 , year=
-
[38]
Xiao, Guangxuan and Lin, Ji and Seznec, Mickael and Wu, Hao and Demouth, Julien and Han, Song , booktitle=
-
[39]
Tim Dettmers and Mike Lewis and Younes Belkada and Luke Zettlemoyer , booktitle=
-
[40]
NVIDIA Blackwell Architecture Technical Brief , author=
-
[41]
NeurIPS , year=
Quarot: Outlier-free 4-bit inference in rotated llms , author=. NeurIPS , year=
-
[42]
Abdelfattah , journal=
Yuzong Chen and Xilai Dai and Jake Hyun and Chi-Chih Chang and Wonsuk Jang and Yuheng Wu and Thierry Tambe and Jae-sun Seo and Mohamed S. Abdelfattah , journal=
-
[43]
NeurIPS , year=
Pytorch: An imperative style, high-performance deep learning library , author=. NeurIPS , year=
-
[44]
NeurIPS , year=
An imperative style, high-performance deep learning library , author=. NeurIPS , year=
-
[45]
ICLR , year=
Pointer sentinel mixture models , author=. ICLR , year=
-
[46]
JMLR , year=
Exploring the limits of transfer learning with a unified text-to-text transformer , author=. JMLR , year=
-
[47]
arXiv preprint arXiv:1803.05457 , year=
Think you have solved question answering? try arc, the ai2 reasoning challenge , author=. arXiv preprint arXiv:1803.05457 , year=
-
[48]
ACL , year=
Hellaswag: Can a machine really finish your sentence? , author=. ACL , year=
-
[49]
AAAI , year=
Piqa: Reasoning about physical commonsense in natural language , author=. AAAI , year=
-
[50]
AAAI , year=
WINOGRANDE: An Adversarial Winograd Schema Challenge at Scale , author=. AAAI , year=
-
[51]
ICLR , year=
Decoupled Weight Decay Regularization , author=. ICLR , year=
-
[52]
EMNLP , year=
Llm-fp4: 4-bit floating-point quantized transformers , author=. EMNLP , year=
-
[53]
arXiv preprint arXiv:2510.25602 , year=
INT vs FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formats , author=. arXiv preprint arXiv:2510.25602 , year=
-
[54]
arXiv preprint arXiv:2110.14168 , year=
Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=
-
[55]
ICLR , year=
Measuring massive multitask language understanding , author=. ICLR , year=
-
[56]
arXiv preprint arXiv:1308.3432 , year=
Estimating or propagating gradients through stochastic neurons for conditional computation , author=. arXiv preprint arXiv:1308.3432 , year=
-
[57]
2023 , month = sep, url =
2023
-
[58]
arXiv preprint arXiv:2605.12245 , year=
SOAR: Scale Optimization for Accurate Reconstruction in NVFP4 Quantization , author=. arXiv preprint arXiv:2605.12245 , year=
-
[59]
ICLR , year=
PT ^2 -LLM: Post-Training Ternarization for Large Language Models , author=. ICLR , year=
-
[60]
ICLR , year=
Arb-llm: Alternating refined binarizations for large language models , author=. ICLR , year=
-
[61]
EMNLP , year=
Transformers: State-of-the-art natural language processing , author=. EMNLP , year=
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.