Pith. sign in

REVIEW 2 major objections 4 minor 61 references

FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling

T0 review · 2 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Pith's one-line read The paper claims that the quantization scale in FP4 LLM quantization can be fully decoupled from the hardware-mandated dequantization scale, and that learning this free scale in full precision recovers nearly all FP16 accuracy without any i

desk verdict A genuinely useful scale-decoupling idea with mostly clean experiments, but a WikiText2-below-FP16 anomaly and a missing appendix mean the SOTA claim needs scrutiny before I'd fully trust it. read the letter →

arxiv 2608.01847 v1 pith:QMGIOGXW submitted 2026-08-03 cs.AI

classification cs.AI
keywords FP4quantizationpost-trainingscaleoptimizationMXFP4NVFP4largelanguagemodelscoupled-relaxationscalingdual-granularity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FOCUS is a post-training quantization method for FP4 formats that challenges a hidden assumption: the scale used to quantize weights offline does not need to follow the same precision or block-size rules as the scale stored for deployment. The paper argues that only the dequantization scale is kept at inference, so the quantization scale can be a learnable full-precision value, optionally refined per sub-block. If correct, this gives state-of-the-art accuracy under both MXFP4 and NVFP4 formats while leaving the deployed model in standard hardware format with zero extra inference cost. Practitioners could then run FP4 LLMs at near-FP16 quality without changing their inference stack.

What carries the argument

The key object is the pair (S^q, S^{dq})—the quantization and dequantization scales. Standard methods enforce S^q = S^{dq} = Q_fmt(S^{fp}), where Q_fmt is the low-precision scale format (E8M0 for MXFP4, E4M3 for NVFP4). FOCUS relaxes this to S^q = S^{dq} * c, with c a learnable full-precision coefficient (Coupled-Relaxation Scaling), and further splits each hardware block into sub-blocks that each get their own coefficient (Dual-Granularity Scaling). This lifts scale optimization from a discrete, hardware-constrained space to a continuous, gradient-optimizable space while keeping the deployed S^{dq} fully compliant. The learning loop uses straight-through estimation for non-differentiable qu

What would settle it

Calibrate FOCUS on WikiText2 and evaluate on a held-out corpus distinct from WikiText2 and C4 (e.g., a code dataset or a different-domain benchmark); if the perplexity or accuracy advantage over a hardware-constrained baseline like MR-GPTQ disappears or reverses, the free-scale optimization is overfitting to the calibration distribution rather than finding genuinely better FP4 assignments.

Watch

Extended reading notes

Core claim

The central discovery is that the two scales in FP4 quantization serve different roles and need not be equal: the dequantization scale is stored and must conform to the hardware format and block size, but the quantization scale is only used offline to assign FP4 values and is never retained. FOCUS exploits this asymmetry by expressing the quantization scale as the dequantization scale times a learnable full-precision coefficient (Coupled-Relaxation Scaling), and by letting that coefficient vary per sub-block within a hardware block (Dual-Granularity Scaling). Because the coefficients are discarded after calibration, the deployed model remains standard MXFP4/NVFP4 format with no additional in

Load-bearing premise

The method's gains rest on the assumption that the 1,248-sample WikiText2 calibration set, with a sequence length of 2,048, is representative enough that the learned scale coefficients transfer to other text distributions, zero-shot reasoning tasks, and multi-step math reasoning.

Editorial extensions

If this is right

  • FP4 quantization can be improved through gradient-based scale learning without adding any runtime cost, since only the standard FP4 weights and dequantization scales are stored.
  • MXFP4, whose E8M0 scales are inherently coarse, benefits enough to approach FP16 language modeling perplexity, making the cheaper format more viable for deployment.
  • The relaxation trick is complementary to transform-based methods (e.g., Hadamard rotations), so future pipelines can combine offline scale relaxation with online transforms for further accuracy.
  • Quantization time remains practical for post-training use—about 19 minutes on a single GPU for a 4-billion-parameter model—making the method a drop-in replacement for existing scale-search steps.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the quantization scale is truly unconstrained, the same decoupling idea could apply to other low-precision formats such as FP8 or MXFP6, where dequantization scales are also format-limited, potentially yielding similar accuracy gains.
  • The reported WikiText2 perplexity for Qwen3-4B MXFP4 is lower than the FP16 baseline, which hints that the learned scales may be overfit to the calibration distribution; a different calibration corpus or evaluation split would test whether the gains generalize.
  • The sub-block coefficients effectively act as per-sub-block rounding-boundary shifts; this suggests a broader design space where offline quantization parameters can be tuned for purposes other than minimizing reconstruction error, such as preserving downstream task logits.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes FOCUS, a post-training quantization method for FP4 formats (MXFP4 and NVFP4). Its central observation is that the quantization scale used offline to map weights to FP4 is never stored and therefore need not satisfy the hardware precision/block-size constraints imposed on the dequantization scale. FOCUS implements this via Coupled-Relaxation Scaling (CRS), a learnable full-precision coefficient multiplying the hardware-compliant dequantization scale, and Dual-Granularity Scaling (DGS), which applies independent sub-block coefficients during quantization while leaving the deployed format unchanged. Experiments on Qwen3 and LLaMA-3.1/3.2 models report state-of-the-art FP4 perplexity and zero-shot accuracy under both formats, with a concrete cost comparison and no inference-time overhead.

Significance. If the empirical claims survive scrutiny, FOCUS is a conceptually clean and practically useful contribution: it exploits a genuinely unused degree of freedom (the offline-only quantization scale), preserves hardware compatibility, and is orthogonal to transform-based methods. The ablations cleanly separate CRS and DGS, the quantization-cost table is concrete, and the authors commit to releasing code. The main risk is that the learned scale/coefficient optimization is heavily overparameterized relative to the calibration data; Table 1 shows a calibration-specialization signature (WikiText2 perplexity below FP16 while C4 remains above FP16). The missing appendix also prevents verification of several claims that motivate the design and the comparison. These are addressable, so the central idea is defensible pending stronger generalization evidence.

major comments (2)
  1. [Experiments, Tables 1 and 4] The Qwen3-4B results show FOCUS below the FP16 reference on WikiText2 (12.85 MXFP4, 12.57 NVFP4 vs 13.66 FP16) while remaining above FP16 on C4 (17.77/16.97 vs 16.63). Table 4 shows even the naive scale-learning baseline already reaches 12.96 on Wiki2. Since calibration uses 1,248 WikiText2 training sequences (~2.5M tokens) and the optimized parameters number roughly 0.16-0.19 per element (~6e8 for a 4B model), this pattern is consistent with specialization to the calibration distribution rather than a generally better FP4 representation. The SOTA claim spans C4, zero-shot, and reasoning benchmarks, so the paper should demonstrate cross-corpus generalization: calibrate on C4 (or a second corpus), evaluate on multiple held-out sets, and report calibration-seed/split variance. Without this, the headline numbers do not establish a general accuracy improvement.
  2. [Methodology/CRS and 'FOCUS without Extra Transforms'; missing Appendix] Several load-bearing results are deferred to an appendix that is not present in the submitted text: the claimed full-decoupling training instability that motivates CRS; the Hadamard fusion experiments and re-quantization degradation; the negligible additional training cost; and the claimed complementarity/combinations with transform-based methods. The CRS design rationale and the comparison to transform-based SOTA cannot be checked without these results. The authors should either provide the appendix in full or remove/qualify the claims that depend on it.
minor comments (4)
  1. [Experimental Settings] The KL-TopLoss objective with k=1000 is referenced to Hu et al. but not defined; please specify the exact loss (e.g., top-k token KL divergence) and how k affects optimization.
  2. [Table 3] GSM8K/MMLU CoT results are reported only for MXFP4, although the abstract claims accuracy under both formats. Please add NVFP4 reasoning results or qualify the claim.
  3. [Related Work] Related Work credits SOAR (Bao et al. 2026) with 'decoupled scale search'; the introduction's statement that existing methods 'tightly couple' quantization and dequantization scales should be qualified to avoid an apparent contradiction.
  4. [Tables and text] There are formatting/spacing issues (e.g., '12.8517.77' in Table 5 and missing spaces in phrases such as 'FOCUSiscomplementary' and 'Sq i' constructions). Please proofread the final version.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FOCUS's central claims are supported by independent held-out evaluation; the overlapping-author SOAR citation is not load-bearing.

full rationale

The paper's derivation chain does not reduce to its inputs. The key premise that the quantization scale is free from hardware constraints follows directly from the deployment description in Eqs. (5)-(7): Sq is used only offline and discarded, while Sdq is stored, so only Sdq must conform to E4M3/E8M0. This is an observation about the method's own forward pass, not a conclusion imported from a self-citation. CRS and DGS are validated by training the relaxation coefficients on a WikiText2 calibration subset and measuring generalization on held-out C4, zero-shot tasks, GSM8K, and MMLU; no fitted scale or coefficient is relabeled as a prediction. The overlapping-author citation (Bao et al. 2026, SOAR) appears only as related-work context and is not used to justify FOCUS's claims or to rule out alternatives; there is no imported uniqueness theorem and no ansatz smuggled in via citation. Several details are deferred to a missing appendix (full-decoupling instability, Hadamard-fusion experiments, transform combination), which is a verifiability/completeness concern rather than a circularity concern. Overall, the empirical results stand as external evidence, so the circularity score is 0.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim depends on learned scale parameters and standard ML assumptions; no new physical or mathematical entities are introduced.

free parameters (6)
  • Relaxation coefficient c_i (CRS) = learned, init=1
    Full-precision multiplier between quantization and dequantization scales, optimized end-to-end; core contribution.
  • Sub-block coefficients c^k_i (DGS) = learned, init=1
    Per-sub-block multipliers for quantization scale, discarded after calibration.
  • Scale learning rates = MXFP4: 2e-2 (scale), 5e-2 (coef); NVFP4: 5e-3 (scale), 1e-3 (coef)
    Hand-chosen hyperparameters for optimization.
  • Sub-block size = 8 for both formats
    Selected from ablation (Table 5) as a balance of accuracy and cost.
  • KL-TopLoss k = 1000
    Adopted from OSTQuant (Hu et al. 2025) as distillation objective.
  • Calibration samples = 1248 WikiText2 training sequences
    Calibration set size; standard PTQ practice but affects results.
assumptions (5)
  • domain assumption Straight-Through Estimator provides usable gradients through non-differentiable quantization
    Used throughout to train scale parameters (Methodology, Eq. 4 and following).
  • domain assumption FP4 formats (MXFP4 and NVFP4) are correctly represented by the block-scaling equations in Eq. 1-3
    The paper relies on the OCP microscaling specification and NVIDIA NVFP4.
  • domain assumption The quantization scale is never stored at inference
    Key premise of the paper, stated in the introduction; true for the cited formats.
  • domain assumption Knowledge distillation loss (KL-TopLoss) is an effective surrogate for task-level accuracy
    Used as training objective, following OSTQuant.
  • domain assumption Calibration data (WikiText2 training) is representative of evaluation distributions
    This is the weakest link, see red flags.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling." pith.science (2026). https://pith.science/paper/QMGIOGXW

@misc{pith2026260801847,
  author       = {Pith},
  title        = {Pith review of: FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QMGIOGXW}},
  note         = {Machine review of arXiv:2608.01847}
}
read the original abstract

Large language models (LLMs) achieve remarkable performance but are expensive to deploy due to their enormous size. FP4 quantization, with formats such as MXFP4 and NVFP4, offers an appealing solution with native hardware support on modern accelerators. However, maintaining accuracy under FP4 precision remains difficult. A key bottleneck lies in scale optimization: existing methods tightly couple the quantization and dequantization scales, forcing both to conform to the discrete low-precision format required by hardware, such as E8M0 in MXFP4. Yet the quantization scale is never stored and need not obey this constraint, suggesting a significant untapped optimization space. In this work, we propose FOCUS, a post-training quantization framework with end-to-end scale learning for FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling. Coupled-Relaxation Scaling (CRS) relaxes the tight coupling between quantization and dequantization scales with a learnable full-precision coefficient, enabling more effective optimization without breaking hardware compliance. Dual-Granularity Scaling (DGS) further refines the quantization scale at a finer sub-block granularity, allowing more precise adaptation to local weight distributions. Experiments across multiple LLM families and benchmarks show that FOCUS achieves state-of-the-art FP4 accuracy under both MXFP4 and NVFP4 formats, while introducing no additional inference overhead. Code and quantized models will be released at https://github.com/tencent/AngelSlim.

Figures

Figures reproduced from arXiv: 2608.01847 by the authors.

Figure 1
Figure 1. FP4 quantization performance on Qwen3-4B. Recovery rate is the average zero-shot accuracy relative to FP16. Our method consistently outperforms all baselines under both NVFP4 and MXFP4 formats. pushed from 8-bit (Dettmers et al. 2022) to 4-bit (Ashkboos et al. 2024) and even lower precision (Yan et al. 2026; Li et al. 2025). Recently, microscaling floating-point formats such as MXFP4 (Rouhani et al. 2023) and NVFP4 … view at source ↗
Figure 2
Figure 2. Standard FP4 quantization vs. FOCUS. (a) Con￾ventional methods tightly couple Sq and Sdq, constraining both to the same low-precision format and block granularity. (b) FOCUS relaxes this coupling, freeing the quantization scale from the hardware-imposed precision and granularity constraints. rules and yield limited improvements. Overall, scale opti￾mization for FP4 formats remains largely underexplored in current LL… view at source ↗
Figure 3
Figure 3. Overview of FOCUS. (a) End-to-end Scale Learning Baseline: learns FP8 scales via STE while weights are frozen during quantization and dequantization. (b) Coupled-Relaxation Scaling (CRS): introduces learnable relaxation coefficients for additional flexibility beyond FP8 precision. (c) Dual-Granularity Scaling (DGS): partitions each block into sub-blocks with independent coefficients for fine-grained scale adjustment… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Sub-block partition in Dual-Granularity Scaling for [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

61 extracted references · 12 linked inside Pith

  1. [1]

    Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education

    Clancey, William J. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83)

  2. [2]

    Classification Problem Solving

    Clancey, William J. Classification Problem Solving. Proceedings of the Fourth National Conference on Artificial Intelligence

  3. [3]

    , title =

    Robinson, Arthur L. , title =. 1980 , doi =. https://science.sciencemag.org/content/208/4447/1019.full.pdf , journal =

  4. [4]

    New Ways to Make Microcircuits Smaller---Duplicate Entry

    Robinson, Arthur L. New Ways to Make Microcircuits Smaller---Duplicate Entry. Science

  5. [5]

    Clancey and Glenn Rennels , abstract =

    Diane Warner Hasling and William J. Clancey and Glenn Rennels , abstract =. Strategic explanations for a diagnostic consultation system , journal =. 1984 , issn =. doi:https://doi.org/10.1016/S0020-7373(84)80003-6 , url =

  6. [6]

    and Rennels, Glenn R

    Hasling, Diane Warner and Clancey, William J. and Rennels, Glenn R. and Test, Thomas. Strategic Explanations in Consultation---Duplicate. The International Journal of Man-Machine Studies

  7. [7]

    Poligon: A System for Parallel Problem Solving

    Rice, James. Poligon: A System for Parallel Problem Solving

  8. [8]

    Transfer of Rule-Based Expertise through a Tutorial Dialogue

    Clancey, William J. Transfer of Rule-Based Expertise through a Tutorial Dialogue

Show all 61 references
  1. [9]

    The Engineering of Qualitative Models

    Clancey, William J. The Engineering of Qualitative Models

  2. [10]

    2023 , eprint=

    Attention Is All You Need , author=. 2023 , eprint=

  3. [11]

    Pluto: The 'Other' Red Planet

    NASA. Pluto: The 'Other' Red Planet

  4. [12]

    arXiv preprint arXiv:2512.02010 , year=

    Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling , author=. arXiv preprint arXiv:2512.02010 , year=

  5. [13]

    2023 , booktitle=

    OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models , author=. 2023 , booktitle=

  6. [14]

    LRQuant: Learnable and Robust Post-Training Quantization for Large Language Models , author=

  7. [15]

    SpinQuant: LLM Quantization with Learned Rotations , author=

  8. [16]

    AffineQuant: Affine Transformation Quantization for Large Language Models , author=

  9. [17]

    FlatQuant: Flatness Matters for LLM Quantization , author=

  10. [18]

    Up or Down? Adaptive Rounding for Post-Training Quantization , author=

  11. [19]

    BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction , author=

  12. [20]

    OSTQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting , author=

  13. [21]

    Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization , author=

  14. [22]

    Block Rotation is All You Need for MXFP4 Quantization , author=

  15. [23]

    ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs , author=

  16. [24]

    Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction , author=

  17. [25]

    BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization , author=

  18. [26]

    arXiv preprint arXiv:2310.10537 , year=

    Microscaling Data Formats for Deep Learning , author=. arXiv preprint arXiv:2310.10537 , year=

  19. [27]

    arXiv preprint arXiv:2509.25149 , year=

    Pretraining Large Language Models with NVFP4 , author=. arXiv preprint arXiv:2509.25149 , year=

  20. [28]

    arXiv preprint arXiv:2601.09555 , year=

    Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formats , author=. arXiv preprint arXiv:2601.09555 , year=

  21. [29]

    arXiv preprint arXiv:2604.17789 , year=

    DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantization , author=. arXiv preprint arXiv:2604.17789 , year=

  22. [30]

    arXiv preprint arXiv:2603.22370 , year=

    FAAR: Format-Aware Adaptive Rounding for NVFP4 , author=. arXiv preprint arXiv:2603.22370 , year=

  23. [31]

    Frantar, Elias and Ashkboos, Saleh and Hoefler, Torsten and Alistarh, Dan , booktitle =

  24. [32]

    Lin, Ji and Tang, Jiaming and Tang, Haotian and Yang, Shang and Chen, Wei-Ming and Wang, Wei-Chen and Xiao, Guangxuan and Dang, Xingyu and Gan, Chuang and Han, Song , booktitle=

  25. [33]

    arXiv preprint arXiv:2505.09388 , year =

    Qwen3 Technical Report , author =. arXiv preprint arXiv:2505.09388 , year =

  26. [34]

    arXiv preprint arXiv:2407.21783 , year=

    The llama 3 herd of models , author=. arXiv preprint arXiv:2407.21783 , year=

  27. [35]

    arXiv preprint arXiv:2501.12948 , year=

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

  28. [36]

    arXiv preprint arXiv:2507.20534 , year=

    Kimi k2: Open agentic intelligence , author=. arXiv preprint arXiv:2507.20534 , year=

  29. [37]

    arXiv preprint arXiv:2602.15763 , year=

    Glm-5: from vibe coding to agentic engineering , author=. arXiv preprint arXiv:2602.15763 , year=

  30. [38]

    Xiao, Guangxuan and Lin, Ji and Seznec, Mickael and Wu, Hao and Demouth, Julien and Han, Song , booktitle=

  31. [39]

    Tim Dettmers and Mike Lewis and Younes Belkada and Luke Zettlemoyer , booktitle=

  32. [40]

    NVIDIA Blackwell Architecture Technical Brief , author=

  33. [41]

    NeurIPS , year=

    Quarot: Outlier-free 4-bit inference in rotated llms , author=. NeurIPS , year=

  34. [42]

    Abdelfattah , journal=

    Yuzong Chen and Xilai Dai and Jake Hyun and Chi-Chih Chang and Wonsuk Jang and Yuheng Wu and Thierry Tambe and Jae-sun Seo and Mohamed S. Abdelfattah , journal=

  35. [43]

    NeurIPS , year=

    Pytorch: An imperative style, high-performance deep learning library , author=. NeurIPS , year=

  36. [44]

    NeurIPS , year=

    An imperative style, high-performance deep learning library , author=. NeurIPS , year=

  37. [45]

    ICLR , year=

    Pointer sentinel mixture models , author=. ICLR , year=

  38. [46]

    JMLR , year=

    Exploring the limits of transfer learning with a unified text-to-text transformer , author=. JMLR , year=

  39. [47]

    arXiv preprint arXiv:1803.05457 , year=

    Think you have solved question answering? try arc, the ai2 reasoning challenge , author=. arXiv preprint arXiv:1803.05457 , year=

  40. [48]

    ACL , year=

    Hellaswag: Can a machine really finish your sentence? , author=. ACL , year=

  41. [49]

    AAAI , year=

    Piqa: Reasoning about physical commonsense in natural language , author=. AAAI , year=

  42. [50]

    AAAI , year=

    WINOGRANDE: An Adversarial Winograd Schema Challenge at Scale , author=. AAAI , year=

  43. [51]

    ICLR , year=

    Decoupled Weight Decay Regularization , author=. ICLR , year=

  44. [52]

    EMNLP , year=

    Llm-fp4: 4-bit floating-point quantized transformers , author=. EMNLP , year=

  45. [53]

    arXiv preprint arXiv:2510.25602 , year=

    INT vs FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formats , author=. arXiv preprint arXiv:2510.25602 , year=

  46. [54]

    arXiv preprint arXiv:2110.14168 , year=

    Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=

  47. [55]

    ICLR , year=

    Measuring massive multitask language understanding , author=. ICLR , year=

  48. [56]

    arXiv preprint arXiv:1308.3432 , year=

    Estimating or propagating gradients through stochastic neurons for conditional computation , author=. arXiv preprint arXiv:1308.3432 , year=

  49. [57]

    2023 , month = sep, url =

  50. [58]

    arXiv preprint arXiv:2605.12245 , year=

    SOAR: Scale Optimization for Accurate Reconstruction in NVFP4 Quantization , author=. arXiv preprint arXiv:2605.12245 , year=

  51. [59]

    ICLR , year=

    PT ^2 -LLM: Post-Training Ternarization for Large Language Models , author=. ICLR , year=

  52. [60]

    ICLR , year=

    Arb-llm: Alternating refined binarizations for large language models , author=. ICLR , year=

  53. [61]

    EMNLP , year=

    Transformers: State-of-the-art natural language processing , author=. EMNLP , year=

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.