Pith.

Machine Learning · arXiv:2611.16426 v1 · submitted 12 Nov 2026 · 42 references

SoScale: Second-Order Scale Search for FP4 Quantization

Pengxiang Zhao, Xing Li, Hui-Ling Zhen, Weizhe Lin, Xianzhi Yu, Zhenhua Dong

The paper says

“Low-bit quantization reduces the memory footprint and inference cost of large language models (LLMs), yet accurate FP4 quantization remains challenging.”

more of the abstract

Practical formats such as MXFP4, NVFP4, and HiF4 pair a 4-bit floating-point payload with block-wise or hierarchical scale codes, making accuracy highly sensitive to how each local distribution is scaled. Default maximum-absolute-value scaling preserves dynamic range, but can spend scarce FP4 levels on rarely occupied outlier regions and leave dense tensor values poorly resolved. Existing scale-search methods minimize local element-wise reconstruction error, optimizing tensor fidelity rather than layer-output fidelity. We propose SoScale, a second-order weight scale-code search method for hierarchical FP4 quantization. SoScale searches hardware-valid scale codes under a Hessian-aware layer reconstruction objective and couples this search with GPTQ-style error recovery, selecting scales by their impact on layer outputs. Across Qwen3 models, SoScale consistently reduces downstream quantization degradation under MXFP4, NVFP4, and HiF4.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 13 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

The passes disagreed. Reader: holds up. Skeptic: holds up only with conditions. Referee: major revision.

Solid incremental PTQ work that combines Hessian-aware scale search with GPTQ recovery for FP4 formats; the experiments are clean and the gains are consistent but modest, and the main unaddressed question is whether the block-local Hessian approximation holds for wider architectures. This line is wrong

2 major objections · 5 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 42 references say.

The paper says, at Methodology, Eq. (7) and surrounding text

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.86):

“The ideal choice would minimize the layer recon- structionlossaboveoverallscalecodes,butthefullHessian couples errors across blocks and makes joint search combi- natorial.”

Pith says: Objection 1 of 2 · major

The central claim rests on the block-local Hessian submatrix H_b (Eq. 5) to score scale codes (Eq. 7), ignoring cross-block Hessian couplings. This is acknowledged as a combinatorial limitation (b68), but the paper provides no empirical or theoretical justification that the approximation is safe in practice.

more of objection 1

If cross-block correlations are significant, a block-locally optimal scale could be globally suboptimal. Table 4 ablates the contribution of error recovery but does not isolate the effect of the block-diagonal assumption. I request an ablation study that varies the block size (e.g., merging two or more FP4 blocks) or, on a small layer, compares the full Hessian objective (joint search over a small number of blocks) against the block-local surrogate to demostrate that the approximation does not systematically degrad the search quality.

The paper says, at Methodology, Eq. (9) and Algorithm 1

The review did not quote this passage.

Pith says: Objection 2 of 2 · major

The damping coefficient λ (Eq. 9) is a free parameter, but no sensitivity analysis or default value is reported. Similarly, the offset window used to construct the candidate set C_{i,b} (after Eq. 8) is not speficied. For reproducibility, the manuscript should report the value of λ and the ofset window (e.g., ±1 for MXFP4 E8M0 codes) for each format, and preferably show the impact on downstream accuracy across a range of λ values.

5 minor objections

The paper says, at Table 1

Pith says: minor objection 1 of 5

The row 'Bits / element' for HiF4 is listed as 4.50, but the text (after Eq.4) and Table 1 should clarify that this is the average due to shared hierarchy; a brief footnote or explanation in the caption would help.

The paper says, at Figure 2

Pith says: minor objection 2 of 5

The flowchart is informative but the surrounding text (e.g., the caption and the paragraph explaining the figure) could be more explicit about the flow of activation quantization vs. weight quantization; currently the figure shows activation quantization on the right but the method does not search activation scales.

show 3 more

The paper says, at Table 2 and Table 3

Pith says: minor objection 3 of 5

Table 2 is titled 'Main Results' but the 'Direct activation RTN' results in Table 3 are described in the main text as 'deployment-oriented'; the presentation would be clearer if the primary claim (SoScale improves across all formats) is first supported by Table 2, and Table 3 is clearly marked as a supplementary ablation study.

The paper says, at References

Pith says: minor objection 4 of 5

Several references use 'arXiv:...' without a year or full bibliographic information; the format should be corrected for consistency (e.g., 'arXiv:2605.12464' is missing a year in the textual reference).

The paper says, at Related work

Pith says: minor objection 5 of 5

The sentence 'The closest work adapts scales directly: Four Over Six switches between two NVFP4 rules... and ScaleSearch minimizes local MSE over block scales' could be more precise by noting that Four Over Six is a rule-based switch rather than a search, while ScaleSearch is a search over an enumerable set.

What would settle it

For a representative layer (e.g., the first QKV projection in Qwen3-8B), compute the full Hessian H and the block-diagonal approximation diag(H_b). Measure the Frobenius norm of off-diagonal blocks relative to diagonal blocks. Then quantize that layer using (a) SoScale as-is, (b) a variant that merges adjacent blocks into larger blocks (e.g., 2× block size) for scale search while keeping the FP4 format's original block granularity for encoding. If variant (b) consistently improves accuracy or perplexity over (a), the block-local approximation is lossy and the central argument is weakened.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 13 Nov 2026

  1. Reader holds up confidence high
  2. Skeptic partly agrees: says it holds up only with conditions 19 s · 16,985 tokens read, 2,258 written Block-local second-order surrogate (Eq. 7) ignores cross-block Hessian coupling; the paper acknowledges this is combinatorial but does not validate whether the approximation is accurate in practice.
  3. Editor wrote the verdict above 9 s · 17,819 tokens read, 965 written
  4. Referee major revision: 2 major objections, 5 minor 28 s · 18,847 tokens read, 1,485 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

SoScale is a well-executed incremental contribution. The idea—using a Hessian-weighted reconstruction loss to select FP4 scale codes, interleaved with GPTQ-style error propagation—is a natural extension of existing scale-search methods, as the authors correctly position. The paper does the important thing: it isolates the Hessian contribution with a clean ablation (Table 4), shows that MSE recovery alone doesn't help much, and demonstrates consistent wins across two model scales and three FP4 formats. The numbers aren't dramatic (0.2–1.1 pp), but that's realistic for formats that already work fairly well.

The soft spot the stress-test note flags is real but not fatal. The block-local surrogate ignores cross-block Hessian coupling, and the paper acknowledges this as a combinatorial limitation (b68). What's missing is an empirical check: e.g., merging neighboring blocks to see whether the approximation degrades. That said, the main results still hold because SoScale outperforms baselines on every format–model combination; if the approximation were badly wrong, you'd expect random failures, not consistent gains. So the concern is worth raising but doesn't sink the paper.

A larger practical concern is the narrow model coverage. Only Qwen3-8B and Qwen3-14B are tested. No Llama, no Mistral, no older architectures. The method likely generalizes, but that's an assumption, not a demonstrated fact. Hyperparameters (lambda, offset window) are not fully specified, though reproducibility doesn't depend on them heavily.

Overall, this is a paper for the PTQ audience—people working on deployment of quantized LLMs at 4-bit floating point. It deserves a serious referee. The authors think carefully, the claims are proportionate, and the experiments are honestly reported. I'd bring it to a reading group for the ablation design alone.

The strongest objection

The central claim—that SoScale improves downstream accuracy via Hessian-aware scale search—rests on the block-local surrogate of Eq. 7, which scores each block using only its Hessian submatrix H_b, neglecting off-diagonal couplings between blocks. The paper states this is a combinatorial limitation (b68), but does not provide any empirical or theoretical justification that the approximation is safe. If cross-block correlations are significant, a scale that is locally optimal could be globally suboptimal; the GPTQ-style error recovery (Eq. 13) only compensates errors after scale commitment, not before. Table 4 shows that full SoScale (Hessian + recovery) works, but does not isolate the effect of ignoring cross-block coupling: a variant that uses a larger block (merging neighboring blocks) or a full Hessian on a small layer would reveal whether the approximation degrades performance. Without this check, it is unclear whether the gains come from the Hessian weighting or from other components (e.g., error recovery tuning). The concern is load-bearing because if the block-local assumption fails, the method could underperform on layers with strong inter-block correlations, and the claimed consistency across formats would be less reliable.

The reader's weakest_assumption mentions the block-local surrogate and the calibration set. This concern focuses on the block-local surrogate, which the paper acknowledges but does not validate. The reader's verdict is ACCEPT; I recommend CONDITIONAL because the missing validation of the block-local approximation leaves a gap. The paper should provide evidence that ignoring cross-block correlations is benign, or the claim should be tempered to acknowledge this limitation explicitly. A concrete check (as proposed) would settle whether this concern actually lands. If the check shows negligible loss, the verdict could remain ACCEPT.

The referee's report

If the result holds, SoScale provides a practical, offline, hardware-compatible method to improve FP4 quantization accuracy by treating scale code selection as a layer-reconstruction problem rather than a local tensor fitting problem. The method addresses a real failure mode (outlier-induced scale mismatch) and is shown to be effective across three distinct hierarchical FP4 formats. The consistent gains, while modest (≈0.5-1.1% relative), are meaningful for deployment scenarios that aim to preserve as much model quality as possible under very low bi widths. The format-agnostic formulation and the coupling with existing preprocessing methods (e.g., block Hadamard rotations) make the contribution practically relevant.

Pith's reading of the paper (a machine's summary, not the authors')

Hierarchical FP4 quantization relies on blockwise scale codes that determine where the 16 representable levels fall. Default maximum-absolute-value scaling often wastes levels on outlier regions, leaving dense tensor values poorly resolved. SoScale selects scale codes by minimizing a Hessian-weighted layer reconstruction loss rather than local element-wise error. This approach is coupled with GPTQ-style error propagation, so each block is quantized on weights that already incorporate compensation from earlier errors. Across Qwen3-8B and Qwen3-14B on MXFP4, NVFP4, and HiF4, SoScale achieves the best average accuracy in every model-format group, with the largest gains under MXFP4 where scale codes are coarsest.

What carries the argument. The Hessian-weighted block-local score (Eq.7) that uses the empirical Hessian submatrix $H_b$ to weight scale-code candidates, and the sequential error propagation (Eq.13) that compensates future blocks through the inverse-Hessian factor $U$.

Core claim. The paper identifies that for hierarchical FP4 quantization, the choice of block scale codes significantly affects layer output accuracy, but existing methods optimize local tensor fidelity rather than output-aware objectives. SoScale shows that pairing a Hessian-aware block-local scale search with GPTQ-style error propagation, while preserving the target FP4 format, yields consistent accuracy gains over round-to-nearest, GPTQ, local ScaleSearch, and BlockRotation baselines on Qwen3-8B and Qwen3-14B across MXFP4, NVFP4, and HiF4.

The premise it rests on. The method assumes that the block-local Hessian submatrix captures the coupling within the block and that cross-block interactions can be ignored (acknowledged as combinatorial in the paper), and that the calibration set of 128 sequences of 2048 tokens from WikiText is representative of the deployment distribution.

If the paper is right (Pith's inferences, not the authors')

FP4 quantization of LLMs can be improved without changing hardware formats, by offline scale search.

Scale selection for FP4 should be treated as a layer reconstruction problem, not a local tensor fitting problem.

The method complements preprocessing techniques like block Hadamard rotations; the combination is most beneficial for MXFP4.

The format-agnostic framework can be applied to new hierarchical FP4 formats without redesign.

Applying SoScale to activations (not just weights) could further improve accuracy, though the paper uses a dynamic per-token rule.

The Hessian-aware search may generalize to low-bit floating-point formats beyond the three studied, such as other microscaling formats.

The calibration set (128 sequences from WikiText) may not capture all deployment distributions; testing on diverse domains could reveal when the layer objective fails.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Low-bit quantization reduces the memory footprint and inference cost of large language models (LLMs), yet accurate FP4 quantization remains challenging. Practical formats such as MXFP4, NVFP4, and HiF4 pair a 4-bit floating-point payload with block-wise or hierarchical scale codes, making accuracy highly sensitive to how each local distribution is scaled. Default maximum-absolute-value scaling preserves dynamic range, but can spend scarce FP4 levels on rarely occupied outlier regions and leave dense tensor values poorly resolved. Existing scale-search methods minimize local element-wise reconstruction error, optimizing tensor fidelity rather than layer-output fidelity. We propose SoScale, a second-order weight scale-code search method for hierarchical FP4 quantization. SoScale searches hardware-valid scale codes under a Hessian-aware layer reconstruction objective and couples this search with GPTQ-style error recovery, selecting scales by their impact on layer outputs. Across Qwen3 models, SoScale consistently reduces downstream quantization degradation under MXFP4, NVFP4, and HiF4.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-13

The authors' main claim, as the classifier picked it out 1.00

“Across Qwen3 models, SoScale consistently reduces downstream quantization degradation under MXFP4, NVFP4, and HiF4.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.09 · human participants 0.02 · controlled comparison 0.63 · randomized assignment 0.01 · states quantitative result 0.05 · compares with prior results 0.82 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Low-bit quantization reduces the memory footprint and inference cost of large language models (LLMs), yet accurate FP4 quantization remains challenging. background 0.95 0.05 0.03 asserted 1.00
Practical formats such as MXFP4, NVFP4, and HiF4 pair a 4-bit floating-point payload with block-wise or hierarchical scale codes, making accuracy highly sensitive to how each local distribution is scaled. background 1.00 0.06 0.02 asserted 1.00
Default maximum-absolute-value scaling preserves dynamic range, but can spend scarce FP4 levels on rarely occupied outlier regions and leave dense tensor values poorly resolved. background 0.87 0.08 0.02 asserted 0.98
Existing scale-search methods minimize local element-wise reconstruction error, optimizing tensor fidelity rather than layer-output fidelity. gap 0.77 0.04 0.03 asserted 1.00
We propose SoScale, a second-order weight scale-code search method for hierarchical FP4 quantization. contribution 0.98 0.87 0.10 asserted 0.99
SoScale searches hardware-valid scale codes under a Hessian-aware layer reconstruction objective and couples this search with GPTQ-style error recovery, selecting scales by their impact on layer outputs. method 0.97 0.92 0.18 asserted 1.00
Across Qwen3 models, SoScale consistently reduces downstream quantization degradation under MXFP4, NVFP4, and HiF4. result 1.00 0.94 0.79 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.76

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-13: 1 request, 5,602 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (2)
Figure 1 of the paper
Figure 1. Scale-code alignment and model-level impact. On a real Qwen3-8B weight block, scale search changes the MXFP4
Figure 2 of the paper
Figure 2. Workflow of SoScale for one linear layer. Calibration builds the empirical Hessian and damped inverse-Hessian factor.

Figures are the authors', reproduced from the paper.

References (42)
  1. [1] Clancey, William J. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83) on Pith
  2. [2] Clancey, William J. Classification Problem Solving. Proceedings of the Fourth National Conference on Artificial Intelligence on Pith
  3. [3] Robinson, Arthur L. , title =. 1980 , doi =. https://science.sciencemag.org/content/208/4447/1019.full.pdf , journal = on Pith
  4. [4] Robinson, Arthur L. New Ways to Make Microcircuits Smaller---Duplicate Entry. Science on Pith
  5. [5] Diane Warner Hasling and William J. Clancey and Glenn Rennels , abstract =. Strategic explanations for a diagnostic consultation system , journal =. 1984 , issn =. doi:https://doi.org/10.1016/S0020-7373(84)80003-6 , url = on Pithdoi:10.1016/s0020-7373(84)80003-6
  6. [6] Hasling, Diane Warner and Clancey, William J. and Rennels, Glenn R. and Test, Thomas. Strategic Explanations in Consultation---Duplicate. The International Journal of Man-Machine Studies on Pith
  7. [7] Rice, James. Poligon: A System for Parallel Problem Solving on Pith
  8. [8] Clancey, William J. Transfer of Rule-Based Expertise through a Tutorial Dialogue on Pith
  9. [9] Clancey, William J. The Engineering of Qualitative Models on Pith
  10. [10] Attention Is All You Need , author=. 2023 , eprint= on Pith
  11. [11] NASA. Pluto: The 'Other' Red Planet on Pith
  12. [12] Bengio, Yoshua and LeCun, Yann , booktitle =. Scaling Learning Algorithms Towards on Pith
  13. [13] Hinton, Geoffrey E. and Osindero, Simon and Teh, Yee Whye , journal =. A Fast Learning Algorithm for Deep Belief Nets , volume = on Pith
  14. [14] Deep learning , author=. 2016 , publisher= on Pith
  15. [15] DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence , author= on Pith
  16. [16] GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers , author=. 2023 , eprint= on Pith
  17. [17] Awq: Activation-aware weight quantization for on-device llm compression and acceleration , author=. Proceedings of machine learning and systems , volume= on Pith
  18. [18] Smoothquant: Accurate and efficient post-training quantization for large language models , author=. International conference on machine learning , pages=. 2023 , organization= on Pith
  19. [19] Omniquant: Omnidirectionally calibrated quantization for large language models , author=. International Conference on Learning Representations , volume= on Pith
  20. [20] GANQ: GPU-Adaptive Non-Uniform Quantization for Large Language Models , author=. International Conference on Machine Learning , pages=. 2025 , organization= on Pith
show 22 more
  1. [21] Microscaling data formats for deep learning , author=. arXiv preprint arXiv:2310.10537 , year= on PithPith's reviewarXiv
  2. [22] Alvarez, Eduardo and Almog, Omri and Chung, Eric and Layton, Simon and Stosic, Dusan and Krashinsky, Ronny and Aubrey, Kyle , title =. 2025 , month = jun, howpublished = on Pith
  3. [23] HiFloat4 Format for Language Model Inference , author=. 2026 , eprint= on Pith
  4. [24] Block Rotation is All You Need for MXFP4 Quantization , author=. 2025 , eprint= on Pith
  5. [25] Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization , author=. 2026 , eprint= on Pith
  6. [26] ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs , author=. 2026 , eprint= on Pith
  7. [27] Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling , author=. 2026 , eprint= on Pith
  8. [28] Search Your Block Floating Point Scales! , author=. 2026 , eprint= on Pith
  9. [29] Qwen3 Technical Report , author=. 2025 , eprint= on Pith
  10. [30] SpinQuant: LLM quantization with learned rotations , author=. 2025 , eprint= on Pith
  11. [31] QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs , author=. 2024 , eprint= on Pith
  12. [32] Pointer Sentinel Mixture Models , author=. 2016 , eprint= on Pith
  13. [33] Transformers: State-of-the-Art Natural Language Processing , author =. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations , pages = on Pith
  14. [34] Datasets: A Community Library for Natural Language Processing , author =. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations , pages = on Pith
  15. [35] Paszke, Adam and Gross, Sam and Massa, Francisco and Lerer, Adam and Bradbury, James and Chanan, Gregory and Killeen, Trevor and Lin, Zeming and Gimelshein, Natalia and Antiga, Luca and Desmaison, Alban and Kopf, Andreas and Yang, Edward and DeVito, Zachary and Raison, Martin ... on Pith
  16. [36] Training Verifiers to Solve Math Word Problems , author =. 2021 , eprint = on Pith
  17. [37] Zellers, Rowan and Holtzman, Ari and Bisk, Yonatan and Farhadi, Ali and Choi, Yejin , booktitle = on Pith
  18. [38] Measuring Massive Multitask Language Understanding , author =. International Conference on Learning Representations , year = on Pith
  19. [39] Clark, Peter and Cowhey, Isaac and Etzioni, Oren and Khot, Tushar and Sabharwal, Ashish and Schoenick, Carissa and Tafjord, Oyvind , year =. Think you have Solved Question Answering? Try. 1803.05457 , archivePrefix = on PithPith's reviewarXiv
  20. [40] Clark, Christopher and Lee, Kenton and Chang, Ming-Wei and Kwiatkowski, Tom and Collins, Michael and Toutanova, Kristina , booktitle = on Pith
  21. [41] Bisk, Yonatan and Zellers, Rowan and Le Bras, Ronan and Gao, Jianfeng and Choi, Yejin , booktitle = on Pith
  22. [42] Gao, Leo and Tow, Jonathan and Abbasi, Baber and Biderman, Stella and Black, Sid and DiPofi, Anthony and Foster, Charles and Golding, Laurence and Hsu, Jeffrey and Le Noac'h, Alain and Li, Haonan and McDonell, Kyle and Muennighoff, Niklas and Ociepa, Chris and Phang, Jason and... on Pithdoi:10.5281/zenodo.12608602

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment