Machine Learning · arXiv:2611.16426 v1 · submitted 12 Nov 2026 · 42 references
SoScale: Second-Order Scale Search for FP4 Quantization
The paper says
“Low-bit quantization reduces the memory footprint and inference cost of large language models (LLMs), yet accurate FP4 quantization remains challenging.”
more of the abstract
Practical formats such as MXFP4, NVFP4, and HiF4 pair a 4-bit floating-point payload with block-wise or hierarchical scale codes, making accuracy highly sensitive to how each local distribution is scaled. Default maximum-absolute-value scaling preserves dynamic range, but can spend scarce FP4 levels on rarely occupied outlier regions and leave dense tensor values poorly resolved. Existing scale-search methods minimize local element-wise reconstruction error, optimizing tensor fidelity rather than layer-output fidelity. We propose SoScale, a second-order weight scale-code search method for hierarchical FP4 quantization. SoScale searches hardware-valid scale codes under a Hessian-aware layer reconstruction objective and couples this search with GPTQ-style error recovery, selecting scales by their impact on layer outputs. Across Qwen3 models, SoScale consistently reduces downstream quantization degradation under MXFP4, NVFP4, and HiF4.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 13 Nov 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
The passes disagreed. Reader: holds up. Skeptic: holds up only with conditions. Referee: major revision.
Solid incremental PTQ work that combines Hessian-aware scale search with GPTQ recovery for FP4 formats; the experiments are clean and the gains are consistent but modest, and the main unaddressed question is whether the block-local Hessian approximation holds for wider architectures. This line is wrong
2 major objections · 5 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 42 references say.
The paper says, at Methodology, Eq. (7) and surrounding text
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.86):
“The ideal choice would minimize the layer recon- structionlossaboveoverallscalecodes,butthefullHessian couples errors across blocks and makes joint search combi- natorial.”
Pith says: Objection 1 of 2 · major
The central claim rests on the block-local Hessian submatrix H_b (Eq. 5) to score scale codes (Eq. 7), ignoring cross-block Hessian couplings. This is acknowledged as a combinatorial limitation (b68), but the paper provides no empirical or theoretical justification that the approximation is safe in practice.
more of objection 1
If cross-block correlations are significant, a block-locally optimal scale could be globally suboptimal. Table 4 ablates the contribution of error recovery but does not isolate the effect of the block-diagonal assumption. I request an ablation study that varies the block size (e.g., merging two or more FP4 blocks) or, on a small layer, compares the full Hessian objective (joint search over a small number of blocks) against the block-local surrogate to demostrate that the approximation does not systematically degrad the search quality.
The paper says, at Methodology, Eq. (9) and Algorithm 1
The review did not quote this passage.
Pith says: Objection 2 of 2 · major
The damping coefficient λ (Eq. 9) is a free parameter, but no sensitivity analysis or default value is reported. Similarly, the offset window used to construct the candidate set C_{i,b} (after Eq. 8) is not speficied. For reproducibility, the manuscript should report the value of λ and the ofset window (e.g., ±1 for MXFP4 E8M0 codes) for each format, and preferably show the impact on downstream accuracy across a range of λ values.
5 minor objections
The paper says, at Table 1
Pith says: minor objection 1 of 5
The row 'Bits / element' for HiF4 is listed as 4.50, but the text (after Eq.4) and Table 1 should clarify that this is the average due to shared hierarchy; a brief footnote or explanation in the caption would help.
The paper says, at Figure 2
Pith says: minor objection 2 of 5
The flowchart is informative but the surrounding text (e.g., the caption and the paragraph explaining the figure) could be more explicit about the flow of activation quantization vs. weight quantization; currently the figure shows activation quantization on the right but the method does not search activation scales.
show 3 more
The paper says, at Table 2 and Table 3
Pith says: minor objection 3 of 5
Table 2 is titled 'Main Results' but the 'Direct activation RTN' results in Table 3 are described in the main text as 'deployment-oriented'; the presentation would be clearer if the primary claim (SoScale improves across all formats) is first supported by Table 2, and Table 3 is clearly marked as a supplementary ablation study.
The paper says, at References
Pith says: minor objection 4 of 5
Several references use 'arXiv:...' without a year or full bibliographic information; the format should be corrected for consistency (e.g., 'arXiv:2605.12464' is missing a year in the textual reference).
The paper says, at Related work
Pith says: minor objection 5 of 5
The sentence 'The closest work adapts scales directly: Four Over Six switches between two NVFP4 rules... and ScaleSearch minimizes local MSE over block scales' could be more precise by noting that Four Over Six is a rule-based switch rather than a search, while ScaleSearch is a search over an enumerable set.
What would settle it
For a representative layer (e.g., the first QKV projection in Qwen3-8B), compute the full Hessian H and the block-diagonal approximation diag(H_b). Measure the Frobenius norm of off-diagonal blocks relative to diagonal blocks. Then quantize that layer using (a) SoScale as-is, (b) a variant that merges adjacent blocks into larger blocks (e.g., 2× block size) for scale search while keeping the FP4 format's original block granularity for encoding. If variant (b) consistently improves accuracy or perplexity over (a), the block-local approximation is lossy and the central argument is weakened.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 13 Nov 2026
- Reader holds up
- Skeptic partly agrees: says it holds up only with conditions Block-local second-order surrogate (Eq. 7) ignores cross-block Hessian coupling; the paper acknowledges this is combinatorial but does not validate whether the approximation is accurate in practice.
- Editor wrote the verdict above
- Referee major revision: 2 major objections, 5 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
SoScale is a well-executed incremental contribution. The idea—using a Hessian-weighted reconstruction loss to select FP4 scale codes, interleaved with GPTQ-style error propagation—is a natural extension of existing scale-search methods, as the authors correctly position. The paper does the important thing: it isolates the Hessian contribution with a clean ablation (Table 4), shows that MSE recovery alone doesn't help much, and demonstrates consistent wins across two model scales and three FP4 formats. The numbers aren't dramatic (0.2–1.1 pp), but that's realistic for formats that already work fairly well.
The soft spot the stress-test note flags is real but not fatal. The block-local surrogate ignores cross-block Hessian coupling, and the paper acknowledges this as a combinatorial limitation (b68). What's missing is an empirical check: e.g., merging neighboring blocks to see whether the approximation degrades. That said, the main results still hold because SoScale outperforms baselines on every format–model combination; if the approximation were badly wrong, you'd expect random failures, not consistent gains. So the concern is worth raising but doesn't sink the paper.
A larger practical concern is the narrow model coverage. Only Qwen3-8B and Qwen3-14B are tested. No Llama, no Mistral, no older architectures. The method likely generalizes, but that's an assumption, not a demonstrated fact. Hyperparameters (lambda, offset window) are not fully specified, though reproducibility doesn't depend on them heavily.
Overall, this is a paper for the PTQ audience—people working on deployment of quantized LLMs at 4-bit floating point. It deserves a serious referee. The authors think carefully, the claims are proportionate, and the experiments are honestly reported. I'd bring it to a reading group for the ablation design alone.
The strongest objection
The central claim—that SoScale improves downstream accuracy via Hessian-aware scale search—rests on the block-local surrogate of Eq. 7, which scores each block using only its Hessian submatrix H_b, neglecting off-diagonal couplings between blocks. The paper states this is a combinatorial limitation (b68), but does not provide any empirical or theoretical justification that the approximation is safe. If cross-block correlations are significant, a scale that is locally optimal could be globally suboptimal; the GPTQ-style error recovery (Eq. 13) only compensates errors after scale commitment, not before. Table 4 shows that full SoScale (Hessian + recovery) works, but does not isolate the effect of ignoring cross-block coupling: a variant that uses a larger block (merging neighboring blocks) or a full Hessian on a small layer would reveal whether the approximation degrades performance. Without this check, it is unclear whether the gains come from the Hessian weighting or from other components (e.g., error recovery tuning). The concern is load-bearing because if the block-local assumption fails, the method could underperform on layers with strong inter-block correlations, and the claimed consistency across formats would be less reliable.
The reader's weakest_assumption mentions the block-local surrogate and the calibration set. This concern focuses on the block-local surrogate, which the paper acknowledges but does not validate. The reader's verdict is ACCEPT; I recommend CONDITIONAL because the missing validation of the block-local approximation leaves a gap. The paper should provide evidence that ignoring cross-block correlations is benign, or the claim should be tempered to acknowledge this limitation explicitly. A concrete check (as proposed) would settle whether this concern actually lands. If the check shows negligible loss, the verdict could remain ACCEPT.
The referee's report
If the result holds, SoScale provides a practical, offline, hardware-compatible method to improve FP4 quantization accuracy by treating scale code selection as a layer-reconstruction problem rather than a local tensor fitting problem. The method addresses a real failure mode (outlier-induced scale mismatch) and is shown to be effective across three distinct hierarchical FP4 formats. The consistent gains, while modest (≈0.5-1.1% relative), are meaningful for deployment scenarios that aim to preserve as much model quality as possible under very low bi widths. The format-agnostic formulation and the coupling with existing preprocessing methods (e.g., block Hadamard rotations) make the contribution practically relevant.
Pith's reading of the paper (a machine's summary, not the authors')
Hierarchical FP4 quantization relies on blockwise scale codes that determine where the 16 representable levels fall. Default maximum-absolute-value scaling often wastes levels on outlier regions, leaving dense tensor values poorly resolved. SoScale selects scale codes by minimizing a Hessian-weighted layer reconstruction loss rather than local element-wise error. This approach is coupled with GPTQ-style error propagation, so each block is quantized on weights that already incorporate compensation from earlier errors. Across Qwen3-8B and Qwen3-14B on MXFP4, NVFP4, and HiF4, SoScale achieves the best average accuracy in every model-format group, with the largest gains under MXFP4 where scale codes are coarsest.
What carries the argument. The Hessian-weighted block-local score (Eq.7) that uses the empirical Hessian submatrix $H_b$ to weight scale-code candidates, and the sequential error propagation (Eq.13) that compensates future blocks through the inverse-Hessian factor $U$.
Core claim. The paper identifies that for hierarchical FP4 quantization, the choice of block scale codes significantly affects layer output accuracy, but existing methods optimize local tensor fidelity rather than output-aware objectives. SoScale shows that pairing a Hessian-aware block-local scale search with GPTQ-style error propagation, while preserving the target FP4 format, yields consistent accuracy gains over round-to-nearest, GPTQ, local ScaleSearch, and BlockRotation baselines on Qwen3-8B and Qwen3-14B across MXFP4, NVFP4, and HiF4.
The premise it rests on. The method assumes that the block-local Hessian submatrix captures the coupling within the block and that cross-block interactions can be ignored (acknowledged as combinatorial in the paper), and that the calibration set of 128 sequences of 2048 tokens from WikiText is representative of the deployment distribution.
If the paper is right (Pith's inferences, not the authors')
FP4 quantization of LLMs can be improved without changing hardware formats, by offline scale search.
Scale selection for FP4 should be treated as a layer reconstruction problem, not a local tensor fitting problem.
The method complements preprocessing techniques like block Hadamard rotations; the combination is most beneficial for MXFP4.
The format-agnostic framework can be applied to new hierarchical FP4 formats without redesign.
Applying SoScale to activations (not just weights) could further improve accuracy, though the paper uses a dynamic per-token rule.
The Hessian-aware search may generalize to low-bit floating-point formats beyond the three studied, such as other microscaling formats.
The calibration set (128 sequences from WikiText) may not capture all deployment distributions; testing on diverse domains could reveal when the layer objective fails.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Low-bit quantization reduces the memory footprint and inference cost of large language models (LLMs), yet accurate FP4 quantization remains challenging. Practical formats such as MXFP4, NVFP4, and HiF4 pair a 4-bit floating-point payload with block-wise or hierarchical scale codes, making accuracy highly sensitive to how each local distribution is scaled. Default maximum-absolute-value scaling preserves dynamic range, but can spend scarce FP4 levels on rarely occupied outlier regions and leave dense tensor values poorly resolved. Existing scale-search methods minimize local element-wise reconstruction error, optimizing tensor fidelity rather than layer-output fidelity. We propose SoScale, a second-order weight scale-code search method for hierarchical FP4 quantization. SoScale searches hardware-valid scale codes under a Hessian-aware layer reconstruction objective and couples this search with GPTQ-style error recovery, selecting scales by their impact on layer outputs. Across Qwen3 models, SoScale consistently reduces downstream quantization degradation under MXFP4, NVFP4, and HiF4.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-13
The authors' main claim, as the classifier picked it out 1.00
“Across Qwen3 models, SoScale consistently reduces downstream quantization degradation under MXFP4, NVFP4, and HiF4.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.09 · human participants 0.02 · controlled comparison 0.63 · randomized assignment 0.01 · states quantitative result 0.05 · compares with prior results 0.82 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Low-bit quantization reduces the memory footprint and inference cost of large language models (LLMs), yet accurate FP4 quantization remains challenging. | background 0.95 | 0.05 | 0.03 | asserted 1.00 |
| Practical formats such as MXFP4, NVFP4, and HiF4 pair a 4-bit floating-point payload with block-wise or hierarchical scale codes, making accuracy highly sensitive to how each local distribution is scaled. | background 1.00 | 0.06 | 0.02 | asserted 1.00 |
| Default maximum-absolute-value scaling preserves dynamic range, but can spend scarce FP4 levels on rarely occupied outlier regions and leave dense tensor values poorly resolved. | background 0.87 | 0.08 | 0.02 | asserted 0.98 |
| Existing scale-search methods minimize local element-wise reconstruction error, optimizing tensor fidelity rather than layer-output fidelity. | gap 0.77 | 0.04 | 0.03 | asserted 1.00 |
| We propose SoScale, a second-order weight scale-code search method for hierarchical FP4 quantization. | contribution 0.98 | 0.87 | 0.10 | asserted 0.99 |
| SoScale searches hardware-valid scale codes under a Hessian-aware layer reconstruction objective and couples this search with GPTQ-style error recovery, selecting scales by their impact on layer outputs. | method 0.97 | 0.92 | 0.18 | asserted 1.00 |
| Across Qwen3 models, SoScale consistently reduces downstream quantization degradation under MXFP4, NVFP4, and HiF4. | result 1.00 | 0.94 | 0.79 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.76
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-13: 1 request, 5,602 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (2)
References (42)
- [1] Clancey, William J. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83) on Pith
- [2] Clancey, William J. Classification Problem Solving. Proceedings of the Fourth National Conference on Artificial Intelligence on Pith
- [3] Robinson, Arthur L. , title =. 1980 , doi =. https://science.sciencemag.org/content/208/4447/1019.full.pdf , journal = on Pith
- [4] Robinson, Arthur L. New Ways to Make Microcircuits Smaller---Duplicate Entry. Science on Pith
- [5] Diane Warner Hasling and William J. Clancey and Glenn Rennels , abstract =. Strategic explanations for a diagnostic consultation system , journal =. 1984 , issn =. doi:https://doi.org/10.1016/S0020-7373(84)80003-6 , url = on Pithdoi:10.1016/s0020-7373(84)80003-6
- [6] Hasling, Diane Warner and Clancey, William J. and Rennels, Glenn R. and Test, Thomas. Strategic Explanations in Consultation---Duplicate. The International Journal of Man-Machine Studies on Pith
- [7] Rice, James. Poligon: A System for Parallel Problem Solving on Pith
- [8] Clancey, William J. Transfer of Rule-Based Expertise through a Tutorial Dialogue on Pith
- [9] Clancey, William J. The Engineering of Qualitative Models on Pith
- [10] Attention Is All You Need , author=. 2023 , eprint= on Pith
- [11] NASA. Pluto: The 'Other' Red Planet on Pith
- [12] Bengio, Yoshua and LeCun, Yann , booktitle =. Scaling Learning Algorithms Towards on Pith
- [13] Hinton, Geoffrey E. and Osindero, Simon and Teh, Yee Whye , journal =. A Fast Learning Algorithm for Deep Belief Nets , volume = on Pith
- [14] Deep learning , author=. 2016 , publisher= on Pith
- [15] DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence , author= on Pith
- [16] GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers , author=. 2023 , eprint= on Pith
- [17] Awq: Activation-aware weight quantization for on-device llm compression and acceleration , author=. Proceedings of machine learning and systems , volume= on Pith
- [18] Smoothquant: Accurate and efficient post-training quantization for large language models , author=. International conference on machine learning , pages=. 2023 , organization= on Pith
- [19] Omniquant: Omnidirectionally calibrated quantization for large language models , author=. International Conference on Learning Representations , volume= on Pith
- [20] GANQ: GPU-Adaptive Non-Uniform Quantization for Large Language Models , author=. International Conference on Machine Learning , pages=. 2025 , organization= on Pith
show 22 more
- [21] Microscaling data formats for deep learning , author=. arXiv preprint arXiv:2310.10537 , year= on PithPith's reviewarXiv
- [22] Alvarez, Eduardo and Almog, Omri and Chung, Eric and Layton, Simon and Stosic, Dusan and Krashinsky, Ronny and Aubrey, Kyle , title =. 2025 , month = jun, howpublished = on Pith
- [23] HiFloat4 Format for Language Model Inference , author=. 2026 , eprint= on Pith
- [24] Block Rotation is All You Need for MXFP4 Quantization , author=. 2025 , eprint= on Pith
- [25] Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization , author=. 2026 , eprint= on Pith
- [26] ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs , author=. 2026 , eprint= on Pith
- [27] Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling , author=. 2026 , eprint= on Pith
- [28] Search Your Block Floating Point Scales! , author=. 2026 , eprint= on Pith
- [29] Qwen3 Technical Report , author=. 2025 , eprint= on Pith
- [30] SpinQuant: LLM quantization with learned rotations , author=. 2025 , eprint= on Pith
- [31] QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs , author=. 2024 , eprint= on Pith
- [32] Pointer Sentinel Mixture Models , author=. 2016 , eprint= on Pith
- [33] Transformers: State-of-the-Art Natural Language Processing , author =. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations , pages = on Pith
- [34] Datasets: A Community Library for Natural Language Processing , author =. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations , pages = on Pith
- [35] Paszke, Adam and Gross, Sam and Massa, Francisco and Lerer, Adam and Bradbury, James and Chanan, Gregory and Killeen, Trevor and Lin, Zeming and Gimelshein, Natalia and Antiga, Luca and Desmaison, Alban and Kopf, Andreas and Yang, Edward and DeVito, Zachary and Raison, Martin ... on Pith
- [36] Training Verifiers to Solve Math Word Problems , author =. 2021 , eprint = on Pith
- [37] Zellers, Rowan and Holtzman, Ari and Bisk, Yonatan and Farhadi, Ali and Choi, Yejin , booktitle = on Pith
- [38] Measuring Massive Multitask Language Understanding , author =. International Conference on Learning Representations , year = on Pith
- [39] Clark, Peter and Cowhey, Isaac and Etzioni, Oren and Khot, Tushar and Sabharwal, Ashish and Schoenick, Carissa and Tafjord, Oyvind , year =. Think you have Solved Question Answering? Try. 1803.05457 , archivePrefix = on PithPith's reviewarXiv
- [40] Clark, Christopher and Lee, Kenton and Chang, Ming-Wei and Kwiatkowski, Tom and Collins, Michael and Toutanova, Kristina , booktitle = on Pith
- [41] Bisk, Yonatan and Zellers, Rowan and Le Bras, Ronan and Gao, Jianfeng and Choi, Yejin , booktitle = on Pith
- [42] Gao, Leo and Tow, Jonathan and Abbasi, Baber and Biderman, Stella and Black, Sid and DiPofi, Anthony and Foster, Charles and Golding, Laurence and Hsu, Jeffrey and Le Noac'h, Alain and Li, Haonan and McDonell, Kyle and Muennighoff, Niklas and Ociepa, Chris and Phang, Jason and... on Pithdoi:10.5281/zenodo.12608602
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.


No comments yet.
Sign in to comment