{"id":"838db0e6-6930-4a80-b3b3-70ed3fabec75","arxiv_id":"2611.16426","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"low","formal_verification":"none","parameter_count":2,"one_line_summary":"SoScale is a second-order scale-code search method for hierarchical FP4 quantization that uses Hessian-aware layer reconstruction to improve accuracy over local scale search.","lead":"SoScale selects scale codes for FP4 quantization by minimizing a Hessian-weighted layer reconstruction loss, combined with GPTQ-style error recovery. It consistently improves accuracy on Qwen3 models across three FP4 formats without changing the target hardware representation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"Block-local second-order surrogate (Eq. 7) ignores cross-block Hessian coupling; the paper acknowledges this is combinatorial but does not validate whether the approximation is accurate in practice.","rationale":"The reader's weakest_assumption mentions the block-local surrogate and the calibration set. This concern focuses on the block-local surrogate, which the paper acknowledges but does not validate. The reader's verdict is ACCEPT; I recommend CONDITIONAL because the missing validation of the block-local approximation leaves a gap. The paper should provide evidence that ignoring cross-block correlations is benign, or the claim should be tempered to acknowledge this limitation explicitly. A concrete check (as proposed) would settle whether this concern actually lands. If the check shows negligible loss, the verdict could remain ACCEPT.","tokens_in":16985,"tokens_out":2258,"duration_ms":18602,"concrete_test":"For a representative layer (e.g., the first QKV projection in Qwen3-8B), compute the full Hessian H and the block-diagonal approximation diag(H_b). Measure the Frobenius norm of off-diagonal blocks relative to diagonal blocks. Then quantize that layer using (a) SoScale as-is, (b) a variant that merges adjacent blocks into larger blocks (e.g., 2× block size) for scale search while keeping the FP4 format's original block granularity for encoding. If variant (b) consistently improves accuracy or perplexity over (a), the block-local approximation is lossy and the central argument is weakened.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim—that SoScale improves downstream accuracy via Hessian-aware scale search—rests on the block-local surrogate of Eq. 7, which scores each block using only its Hessian submatrix H_b, neglecting off-diagonal couplings between blocks. The paper states this is a combinatorial limitation (b68), but does not provide any empirical or theoretical justification that the approximation is safe. If cross-block correlations are significant, a scale that is locally optimal could be globally suboptimal; the GPTQ-style error recovery (Eq. 13) only compensates errors after scale commitment, not before. Table 4 shows that full SoScale (Hessian + recovery) works, but does not isolate the effect of ignoring cross-block coupling: a variant that uses a larger block (merging neighboring blocks) or a full Hessian on a small layer would reveal whether the approximation degrades performance. Without this check, it is unclear whether the gains come from the Hessian weighting or from other components (e.g., error recovery tuning). The concern is load-bearing because if the block-local assumption fails, the method could underperform on layers with strong inter-block correlations, and the claimed consistency across formats would be less reliable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper proposes SoScale, a second-order (Hessian-aware) scale-code search method for hierarchical FP4 quantization of large language models. SoScale operates offline: for each linear layer, it partitions weights into FP4 blocks, scores legal scale codes using a block-local Hessian submatrix weighted reconstruction loss (Eq. 7), and interleaves this search with GPTQ-style error propagation (Eq. 13). The method is format-agnostic and is instantiated for MXFP4, NVFP4, and HiF4. Experimental results on Qwen3-8B and Qwen3-14B show that SoScale variants achieve the best average downstream accuracy across seven tasks in every model-format group, outperforming round-to-nearest, GPTQ, local ScaleSearch, and BlockRotation baselines. Ablation studies confirm that the full SoScale (Hessian-aware search + recovery) beats both MSE-only search and MSE+recovery.","tokens_in":18847,"tokens_out":1485,"duration_ms":27919,"significance":"If the result holds, SoScale provides a practical, offline, hardware-compatible method to improve FP4 quantization accuracy by treating scale code selection as a layer-reconstruction problem rather than a local tensor fitting problem. The method addresses a real failure mode (outlier-induced scale mismatch) and is shown to be effective across three distinct hierarchical FP4 formats. The consistent gains, while modest (≈0.5-1.1% relative), are meaningful for deployment scenarios that aim to preserve as much model quality as possible under very low bi widths. The format-agnostic formulation and the coupling with existing preprocessing methods (e.g., block Hadamard rotations) make the contribution practically relevant.","major_comments":[{"comment":"The central claim rests on the block-local Hessian submatrix H_b (Eq. 5) to score scale codes (Eq. 7), ignoring cross-block Hessian couplings. This is acknowledged as a combinatorial limitation (b68), but the paper provides no empirical or theoretical justification that the approximation is safe in practice. If cross-block correlations are significant, a block-locally optimal scale could be globally suboptimal. Table 4 ablates the contribution of error recovery but does not isolate the effect of the block-diagonal assumption. I request an ablation study that varies the block size (e.g., merging two or more FP4 blocks) or, on a small layer, compares the full Hessian objective (joint search over a small number of blocks) against the block-local surrogate to demostrate that the approximation does not systematically degrad the search quality.","section":"Methodology, Eq. (7) and surrounding text","authors_acknowledge":"The ideal choice would minimize the layer recon- structionlossaboveoverallscalecodes,butthefullHessian couples errors across blocks and makes joint search combi- natorial.","authors_acknowledge_p":0.86},{"comment":"The damping coefficient λ (Eq. 9) is a free parameter, but no sensitivity analysis or default value is reported. Similarly, the offset window used to construct the candidate set C_{i,b} (after Eq. 8) is not speficied. For reproducibility, the manuscript should report the value of λ and the ofset window (e.g., ±1 for MXFP4 E8M0 codes) for each format, and preferably show the impact on downstream accuracy across a range of λ values.","section":"Methodology, Eq. (9) and Algorithm 1","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The row 'Bits / element' for HiF4 is listed as 4.50, but the text (after Eq.4) and Table 1 should clarify that this is the average due to shared hierarchy; a brief footnote or explanation in the caption would help.","section":"Table 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The flowchart is informative but the surrounding text (e.g., the caption and the paragraph explaining the figure) could be more explicit about the flow of activation quantization vs. weight quantization; currently the figure shows activation quantization on the right but the method does not search activation scales.","section":"Figure 2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Table 2 is titled 'Main Results' but the 'Direct activation RTN' results in Table 3 are described in the main text as 'deployment-oriented'; the presentation would be clearer if the primary claim (SoScale improves across all formats) is first supported by Table 2, and Table 3 is clearly marked as a supplementary ablation study.","section":"Table 2 and Table 3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Several references use 'arXiv:...' without a year or full bibliographic information; the format should be corrected for consistency (e.g., 'arXiv:2605.12464' is missing a year in the textual reference).","section":"References","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The sentence 'The closest work adapts scales directly: Four Over Six switches between two NVFP4 rules... and ScaleSearch minimizes local MSE over block scales' could be more precise by noting that Four Over Six is a rule-based switch rather than a search, while ScaleSearch is a search over an enumerable set.","section":"Related work","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"SoScale is a well-executed incremental contribution. The idea—using a Hessian-weighted reconstruction loss to select FP4 scale codes, interleaved with GPTQ-style error propagation—is a natural extension of existing scale-search methods, as the authors correctly position. The paper does the important thing: it isolates the Hessian contribution with a clean ablation (Table 4), shows that MSE recovery alone doesn't help much, and demonstrates consistent wins across two model scales and three FP4 formats. The numbers aren't dramatic (0.2–1.1 pp), but that's realistic for formats that already work fairly well.\n\nThe soft spot the stress-test note flags is real but not fatal. The block-local surrogate ignores cross-block Hessian coupling, and the paper acknowledges this as a combinatorial limitation (b68). What's missing is an empirical check: e.g., merging neighboring blocks to see whether the approximation degrades. That said, the main results still hold because SoScale outperforms baselines on every format–model combination; if the approximation were badly wrong, you'd expect random failures, not consistent gains. So the concern is worth raising but doesn't sink the paper.\n\nA larger practical concern is the narrow model coverage. Only Qwen3-8B and Qwen3-14B are tested. No Llama, no Mistral, no older architectures. The method likely generalizes, but that's an assumption, not a demonstrated fact. Hyperparameters (lambda, offset window) are not fully specified, though reproducibility doesn't depend on them heavily.\n\nOverall, this is a paper for the PTQ audience—people working on deployment of quantized LLMs at 4-bit floating point. It deserves a serious referee. The authors think carefully, the claims are proportionate, and the experiments are honestly reported. I'd bring it to a reading group for the ablation design alone.","headline":"Solid incremental PTQ work that combines Hessian-aware scale search with GPTQ recovery for FP4 formats; the experiments are clean and the gains are consistent but modest, and the main unaddressed question is whether the block-local Hessian approximation holds for wider architectures.","tokens_in":17819,"tokens_out":965,"duration_ms":8773},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"SoScale uses second-order layer information to select FP4 scale codes, consistently outperforming local scale-search methods on Qwen3 models across three hierarchical FP4 formats.","keywords":["post-training quantization","FP4 quantization","hierarchical FP4","scale code search","second-order quantization","large language models","Hessian-aware","GPTQ"],"falsifier":"If a model with stable outlier distributions showed that SoScale underperforms local MSE search on a strong format like NVFP4 (which has finer scale granularity), then the claim that layer-level sensitivity always improves accuracy would be weakened.","tokens_in":19866,"feed_emoji":"","tokens_out":1203,"duration_ms":34856,"temperature":0.7,"pith_summary":"Hierarchical FP4 quantization relies on blockwise scale codes that determine where the 16 representable levels fall. Default maximum-absolute-value scaling often wastes levels on outlier regions, leaving dense tensor values poorly resolved. SoScale selects scale codes by minimizing a Hessian-weighted layer reconstruction loss rather than local element-wise error. This approach is coupled with GPTQ-style error propagation, so each block is quantized on weights that already incorporate compensation from earlier errors. Across Qwen3-8B and Qwen3-14B on MXFP4, NVFP4, and HiF4, SoScale achieves the best average accuracy in every model-format group, with the largest gains under MXFP4 where scale codes are coarsest.","feed_headline":"One search selects FP4 scales that local methods miss, lifting accuracy across three…","feed_subtitle":"SoScale uses layer output sensitivity, not element error, to choose block scales in Qwen3 models, outscoring GPTQ and local searches.","key_machinery":"The Hessian-weighted block-local score (Eq.7) that uses the empirical Hessian submatrix $H_b$ to weight scale-code candidates, and the sequential error propagation (Eq.13) that compensates future blocks through the inverse-Hessian factor $U$.","core_discovery":"The paper identifies that for hierarchical FP4 quantization, the choice of block scale codes significantly affects layer output accuracy, but existing methods optimize local tensor fidelity rather than output-aware objectives. SoScale shows that pairing a Hessian-aware block-local scale search with GPTQ-style error propagation, while preserving the target FP4 format, yields consistent accuracy gains over round-to-nearest, GPTQ, local ScaleSearch, and BlockRotation baselines on Qwen3-8B and Qwen3-14B across MXFP4, NVFP4, and HiF4.","pith_inferences":["Applying SoScale to activations (not just weights) could further improve accuracy, though the paper uses a dynamic per-token rule.","The Hessian-aware search may generalize to low-bit floating-point formats beyond the three studied, such as other microscaling formats.","The calibration set (128 sequences from WikiText) may not capture all deployment distributions; testing on diverse domains could reveal when the layer objective fails."],"forward_implications":["FP4 quantization of LLMs can be improved without changing hardware formats, by offline scale search.","Scale selection for FP4 should be treated as a layer reconstruction problem, not a local tensor fitting problem.","The method complements preprocessing techniques like block Hadamard rotations; the combination is most beneficial for MXFP4.","The format-agnostic framework can be applied to new hierarchical FP4 formats without redesign."],"supporting_citations":[{"why":"Provides the GPTQ second-order compensation method that SoScale adapts for block-level error propagation.","marker":"Frantar et al. 2023"},{"why":"Defines the local ScaleSearch baseline that optimizes block scales under element-wise MSE, which SoScale surpasses.","marker":"Gupta et al. 2026"},{"why":"Provides the BlockRotation preprocessing baseline that SoScale compares against and complements.","marker":"Ashkboos et al. 2024"},{"why":"Defines the MXFP4 format whose scale codes are searched by SoScale.","marker":"Rouhani et al. 2023"},{"why":"Defines the NVFP4 format used in experiments.","marker":"Alvarez et al. 2025"},{"why":"Defines the HiF4 format used in experiments.","marker":"Luo et al. 2026"},{"why":"Provides the Qwen3 model checkpoints that are quantized.","marker":"Yang et al. 2025"}],"fun_headline_variants":["Hessian-aware scale search beats local methods for FP4 quantization","Scale search via layer output sensitivity lifts FP4 accuracy on LLMs","Second-order scale search: better FP4 quantization by targeting layer output","Pairing Hessian search with GPTQ error recovery improves FP4 quantized LLMs","SoScale: FP4 quantization scales chosen by output impact, not element error"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The method assumes that the block-local Hessian submatrix captures the coupling within the block and that cross-block interactions can be ignored (acknowledged as combinatorial in the paper), and that the calibration set of 128 sequences of 2048 tokens from WikiText is representative of the deployment distribution.","fun_headline_variants_meta":{"raw":{"variants":["Hessian-aware scale search beats local methods for FP4 quantization","Scale search via layer output sensitivity lifts FP4 accuracy on LLMs","Second-order scale search: better FP4 quantization by targeting layer output","Pairing Hessian search with GPTQ error recovery improves FP4 quantized LLMs","SoScale: FP4 quantization scales chosen by output impact, not element error"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000615,"raw_usage":{"cost":0.000614768,"is_byok":false,"cost_details":{"upstream_inference_cost":0.000614768,"upstream_inference_prompt_cost":8.8928e-05,"upstream_inference_completions_cost":0.00052584},"total_tokens":2718,"prompt_tokens":840,"completion_tokens":1878,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":256,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":1780}},"tokens_in":840,"tokens_out":1878,"cost_source":"openrouter_usage","duration_ms":15197,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":1780,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-11-13T15:14:40.953648+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"If a model with stable outlier distributions showed that SoScale underperforms local MSE search on a strong format like NVFP4 (which has finer scale granularity), then the claim that layer-level sensitivity always improves accuracy would be weakened.","supporting_citations":[{"cited_title":"2025 , month = jun, howpublished =","cited_arxiv_id":null,"evidence_quote":"Defines the NVFP4 format used in experiments."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}