{"id":"a33c6f8f-b658-42f2-a1cb-f835f316de49","arxiv_id":"2607.24953","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Transposition-invariant 2D block FP4 quantization plus truncation-free scaling and stochastic rounding enables stable end-to-end FP4 LLM training within ~1% of BF16.","lead":"The paper shows that 4-bit LLM training fails mainly because 1D block scales change after matrix transpose, biasing gradients. Square 2D blocks keep scales consistent, and with mixed FP8 attention the method trains models up to 30B nearly as well as BF16.","discovery_kind":"new_method","skeptic_critique":{"model":"moonshotai/kimi-k3","headline":"The paper's central causal claim — that transposition-induced scale mismatch is the dominant FP4 instability source — is never isolated: no controlled 1D-vs-2D ablation holds truncation-free scaling, stochastic rounding, and MXFP8 fixed while changing only block geometry.","rationale":"I read the paper in good faith: the sufficiency half of the claim (this specific recipe trains stably at 1B–30B/100B tokens within ~1.3% of BF16) is adequately supported by the loss curves and tables, modulo single-seed noise and unnamed baselines. Where I diverge from the reader is on what is most load-bearing. The reader flags software emulation; but FP4/FP8 microscaling formats are deterministic digital formats, so emulated arithmetic is bit-identical to native hardware arithmetic (aside from RNG streams for stochastic rounding). Emulation invalidates the *throughput* story — which the paper explicitly disclaims (§4.1, §5, Table 1 \"idealized\") — not the stability/quality story. So that concern doesn't move the verdict much. The genuinely soft spot is internal: the paper's headline diagnosis (\"scale inconsistency induced by tensor transposition\" as the \"fundamental source\" of instability) is never subjected to a controlled test. Every piece of evidence either (a) compares against unnamed, multiply-different \"competing methods\" (Fig. 4), or (b) ablates only the auxiliary components (Fig. 5). The one ablation that would test the thesis — 1D vs 2D geometry with all else fixed — is absent. Until that comparison exists, the paper demonstrates a working recipe but not the correctness of its mechanistic explanation, and the recipe could be re-implemented with 1D blocks and equal success, which would void the novelty core. This keeps the verdict CONDITIONAL (unchanged), but the condition should be the controlled 1D-vs-2D ablation rather than primarily native-hardware confirmation. The MXFP8-is-worse anomaly and missing multi-seed error bars are secondary and reinforce rather than drive the conditionality.","tokens_in":11596,"tokens_out":3178,"duration_ms":108599,"concrete_test":"Train OLMo-1B for 100B tokens under the exact Table 5 configuration, but replace only the 2D 32×32 weight/gradient blocks with 1D 1×32 MXFP4-style blocks, keeping truncation-free scaling, stochastic rounding, MXFP8 Q/K, and all hyperparameters identical. Run 3 seeds each. If the 1D variant's final loss gap vs BF16 lands within ~1.5% (comparable to the reported 1.1%), the transposition-invariance mechanism is not load-bearing and the contribution reduces to a known-ingredients recipe; if it diverges or the gap exceeds ~2–3%, the paper's diagnosis is confirmed. Optionally add a scale-budget-matched 1D 1×1024 arm to separate geometry from metadata rate.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim is that enforcing forward–backward scaling consistency via 2D blocks is *sufficient* (and, per the motivation sections, *necessary-as-diagnosis*) for stable FP4 training. The sufficiency evidence (their full recipe trains stably within ~1% of BF16) is internally consistent. But the mechanistic core — that the 2D transposition-invariant structure is what does the work — is supported only by the motivational Fig. 1 (scale-difference maps, \"effective weight deviation\") and by Fig. 4's comparison against unnamed \"alternative microscaling schemes\" that differ from the proposed method in multiple unspecified ways. The only controlled ablations (Fig. 5) vary truncation-free scaling and stochastic rounding — both shown necessary — but never the block geometry itself. Table 4 varies 2D block *size* (8×8 to 64×64), never 1D vs 2D. This matters because the recipe's other two components are known-strong stabilizers (stochastic rounding and overflow-free scaling are established in cited prior work, e.g., refs [5, 6, 17]). A plausible alternative reading of the results: truncation-free scaling + SR + MXFP8 Q/K would stabilize 1D MXFP4 training equally well, and the transposition-invariance story is a post-hoc rationale. Note also the 2D 32×32 block shares one scale across 1024 values vs 32 for 1D 1×32 — coarser scaling that would normally *hurt*, so a clean win for 2D at matched recipe would be strong evidence for the thesis; the paper simply never runs it. Secondary issue: the MXFP8 variant is consistently slightly worse than pure 2D-FP4 in Tables 2–3 (higher PPL, lower accuracy), which undercuts the \"practical mixed-precision design\" component of the claim; Appendix G concedes this. The reader's flagged concern (software emulation) is real for speedup claims but weak for the stability claim, since emulated E2M1/E4M3 arithmetic is bit-exact against what native tensor cores compute; quality conclusions transfer. The load-bearing gap is internal: the causal abt","agreement_with_reader":"partial"},"referee_report":{"model":"moonshotai/kimi-k3","summary":"The paper identifies transposition-induced scale inconsistency in 1D microscaling formats (MXFP4/NVFP4) as a source of FP4 training instability: after transposition in backpropagation, values are regrouped into different scaling blocks, so forward and backward passes quantize the same tensor with different scales, biasing gradients under the STE. The proposed fix is 2D square-block FP4 quantization (32×32) for weights and gradients, which preserves block membership under transposition, combined with truncation-free scaling (to avoid overflow) and stochastic rounding in the backward pass (for unbiased gradients), plus MXFP8 for query/key projections. Experiments train OLMo-1B, OLMo-7B, and a Qwen 30B MoE on 100B tokens under software emulation, reporting stable convergence within ~0.8–1.6% of BF16 perplexity and ~1 point on downstream benchmarks.","tokens_in":11944,"tokens_out":2641,"duration_ms":48533,"significance":"If the results hold, this is a useful contribution to low-precision LLM training: stable end-to-end FP4 training up to 30B parameters / 100B tokens with ~1% of BF16 would be a practical step beyond FP8, and the transposition-invariance argument for 2D blocks is simple and mechanistically plausible. Notable strengths: a fully specified, reproducible training configuration (Appendix B: optimizer, schedule, batch size, quantization placement per tensor type); an explicit transposition-invariance proof for square blocks (Appendix D); matched hyperparameters across BF16 and FP4 variants so differences are attributable to the format; and evaluation against external BF16 baselines on public benchmarks rather than internal proxies. The paper is also appropriately candid about the software-emulation limitation (§4.1, §5), and its central design prediction — 2D block geometry should matter independently of scaling/rounding choices — is directly falsifiable with one additional experiment.","major_comments":[{"comment":"The paper's central causal claim — that transposition-induced scale inconsistency is the dominant source of FP4 training instability, and that 2D block geometry is what fixes it — is never isolated experimentally. The only controlled ablations (Fig. 5) vary truncation-free scaling and stochastic rounding, both of which are established stabilizers in the cited prior work (e.g., refs [5, 6, 17]). Table 4 varies 2D block *size* (8×8 to 64×64) but never compares 1D vs 2D geometry at a matched recipe. Figure 4 compares against unnamed 'alternative microscaling schemes' whose configurations are unspecified. The load-bearing experiment is missing: 1D MXFP4 (1×32) with identical truncation-free scaling, stochastic rounding, and MXFP8 Q/K vs the proposed 2D-FP4 with the same components. This experiment would be especially informative because a 32×32 block shares one scale across 1024 values versu","section":"§4.2, Fig. 4, Fig. 5, Table 4"},{"comment":"The MXFP8 Q/K variant consistently performs *worse* than the base 2D-FP4 variant across all reported metrics: Table 2 shows larger perplexity degradation for +MXFP8 at every scale (e.g., 1.6% vs 1.1% avg on OLMo-1B; 1.2% vs 0.9% on OLMo-7B), Table 3 shows lower average accuracy at every scale, and §4.2 acknowledges the loss gap 'increases to 1.6%.' Yet §3.2 motivates MXFP8 Q/K as necessary for stability ('directly quantizing Q and K to FP4 introduces errors that... distort the attention distribution'), and Appendix G reframes it as 'a stability-oriented mixed-precision option.' If the FP4-only configuration is uniformly closer to BF16 and equally stable in all reported runs, the case for the MXFP8 component is unsupported by the paper's own evidence. Either (a) show the FP4-Q/K configuration that fails or destabilizes (currently absent — the only divergence shown is 'naive FP4 without mi","section":"§3.2, Table 2, Table 3, Appendix G"},{"comment":"The abstract states 'less than 1.3% degradation in perplexity and downstream accuracy,' but Table 2 reports 1.6% average perplexity degradation for the 2D-FP4+MXFP8 configuration on OLMo-1B, and Fig. 4/§4.2 repeat the 1.6% figure. The claim is only true for the base 2D-FP4 variant. Since the +MXFP8 configuration is presented as the proposed practical design (§3.2, Table 1, Appendix B where 'Query/Key Precision: MXFP8' appears in the default training configuration), the headline number is inconsistent with the paper's own tables. Please reconcile the abstract claim with Table 2, and clarify which configuration the claim refers to. Separately, Appendix B lists 'Query/Key Precision MXFP8' as the default for all models while the main text treats 2D-FP4 (without MXFP8) as 'Ours' — the default configuration should be stated unambiguously.","section":"Abstract, Table 2, Appendix B"},{"comment":"Figure 4 compares the method against 'alternative microscaling schemes' and 'competing methods' that are never named or cited in the caption or text, and no direct numerical comparison against the most relevant prior FP4 training work — MXFP4 training [17], NVFP4 pretraining [1], or Quartet [4] — appears anywhere in Tables 2–3. Given that [1] reports NVFP4 pretraining results and [4] reports native FP4 training, the absence of any controlled comparison to these baselines makes it impossible to assess the claimed improvement ('final loss gap approximately 1.1%, compared to around 2% for competing methods'). At minimum, the baselines in Fig. 4 must be identified, configured, and cited; ideally one named prior recipe should be run under the same token budget and hyperparameters.","section":"Fig. 4, §4.2"}],"minor_comments":[{"comment":"Fig. 1(d) reports 'effective weight deviation' but the quantity is never defined in the text or appendix. Please give a formula.","section":"Fig. 1"},{"comment":"Table 1's 'ideal linear throughput ~3.56×' and 'activation bandwidth ~0.30×' are described as 'idealized'; given that all experiments are software emulation (§4.1) and no kernel measurements exist, consider moving Table 1 to the appendix or clearly labeling it as a projection, since it does not reflect any measured system.","section":"Table 1"},{"comment":"Table 4 reports ranges ('~0.8–1.0', '~0.9–1.1') rather than single measurements, unlike Tables 2–3. Please report actual measured values with the model/token budget at which they were obtained.","section":"Table 4"},{"comment":"The claim that FP4 exceeding BF16 on SciQ/COPA (Table 3) 'may be attributed to a regularization effect induced by quantization noise' is speculative; either provide supporting evidence (e.g., variance over seeds) or remove. Relatedly, no seed variance or error bars are reported anywhere — for gaps as small as 0.8–1.6%, run-to-run variance matters.","section":"Table 3, §4.2"},{"comment":"Notation: Qp and Qn are used in the scale formula (§3.1) before being defined; the E2M1 bounds should be stated explicitly. Also '∆' column header in Table 2 is undefined at first use (defined only in Appendix F).","section":"§3.1, Table 2"},{"comment":"Activations use 1D-FP4 (Appendix B/C) while weights/gradients use 2D — but per the paper's own thesis, activations also participate in transposed backward GEMMs (∇X = ∇Y W). A brief discussion of why the 1D activation layout does not reintroduce the inconsistency the paper identifies would close an apparent gap in the argument.","section":"§3, Appendix C"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The thing to know: they report stable end-to-end FP4 training on dense 1B/7B and a 30B MoE for 100B tokens, within ~1% of BF16 on perplexity and reasoning, via 2D square-block FP4 plus truncation-free scaling, stochastic rounding, and MXFP8 on Q/K.\n\nWhat is actually new is the framing. Prior MX/NVFP4 work already had block scales, SR, and mixed precision. Their distinctive move is naming transpose-induced scale mismatch in 1D blocks as a primary bias source and fixing it with square 2D blocks so forward and backward keep the same scale assignment. Fig. 1 makes that concrete. The multi-scale loss curves, LM tables, and reasoning numbers are internally consistent, and the recipe is simple enough that systems people can try it. Ablations correctly show that overflow-free scaling and SR each matter. For a cost-center problem like pre-training precision, a near-parity 30B result is real signal even under software emulation.\n\nSoft spots in proportion. The sufficiency claim (full recipe works) is fine. The stronger causal claim—that 2D transposition invariance is what does the work—is under-supported. Fig. 5 only ablates scaling and rounding; Table 4 only varies 2D block size; Fig. 4’s “alternative microscaling” baselines are not a clean 1D-vs-2D holdout with SR and truncation-free scaling fixed. Those other knobs are already known stabilizers in the cited line. A matched 1D control would settle whether geometry is load-bearing or partly post-hoc. Secondary and smaller: pure 2D-FP4 slightly beats the MXFP8 Q/K hybrid on their own tables, so the mixed-precision pitch is a bit ahead of the numbers. Emulation is a weak objection for quality/stability if E2M1 arithmetic is bit-exact; it mainly limits speedup claims, which they mostly do not oversell. No code/seeds/error bars is ordinary for this venue tier but still limits how hard you lean on the 1% gap.\n\nMath is standard STE and block-max scales; citation pattern is fair to the MX/FP4 literature. This is for low-precision training and LLM systems readers. It deserves a serious referee. I would engage, and I would ask for the 1D-vs-2D control before treating the mechanism as settled.","headline":"Useful 30B-scale FP4 training result with a clean diagnosis, but the 2D-geometry causal claim is not isolated from known stabilizers.","tokens_in":13194,"tokens_out":615,"would_cite":true,"duration_ms":27901,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Stable FP4 LLM training works once forward and backward passes use the same block scales after transposition.","keywords":["FP4 training","microscaling","2D block quantization","transposition invariance","stochastic rounding","mixed-precision attention","LLM efficiency","forward-backward consistency"],"falsifier":"Train the same OLMo-1B/7B or 30B MoE recipe on native FP4 tensor-core hardware (or a verified bit-accurate emulator) with 1D vs 2D blocks only, and check whether 1D still shows the forward–backward scale mismatch and early divergence while 2D stays within ~1% of BF16 loss and downstream accuracy through 100B tokens.","tokens_in":12722,"feed_emoji":"🔢","tokens_out":950,"duration_ms":20065,"temperature":0.7,"pith_summary":"Training large language models in 4-bit floating point has been unstable even when block-wise microscaling reduces local quantization error. This paper argues the missing piece is consistency: standard 1D blocks regroup values after matrix transpose, so the same weight or gradient is scaled differently on the forward and backward passes and gradients become biased. Square 2D blocks keep each group of values under one shared scale through transposition, and pairing that with truncation-free scales and stochastic rounding keeps gradients unbiased enough to train. With MXFP8 only on query and key projections, dense models up to 7B and a 30B MoE train stably for 100B tokens and stay within about 1% of BF16 on perplexity and reasoning tasks. If that diagnosis is right, practical FP4 training is mainly a consistency problem, not an impossible precision problem.","feed_headline":"FP4 LLM training stabilizes when block scales survive transpose","feed_subtitle":"2D blocks keep forward and backward scales matched, closing most of the gap to BF16 at 30B scale","key_machinery":"Transposition-invariant 2D block FP4 quantization: partition tensors into b×b square blocks that share one scale so a block and its transpose keep the same maximum and the same scale (S(B)=S(B⊤)), eliminating the forward/backward scale mismatch of 1D layouts.","core_discovery":"The central claim is that transposition-induced scale mismatch in 1D microscaling is a primary failure mode of FP4 training, and that enforcing forward–backward scale consistency with 2D square-block FP4 quantization—plus truncation-free scaling, stochastic rounding, and MXFP8 for Q/K—is sufficient for stable end-to-end FP4 training at scales up to 30B parameters and 100B tokens with less than 1.3% degradation versus BF16.","pith_inferences":["Any low-precision path that secretly changes scales under the adjoint (transpose, reshape, gather) may hit the same bias even outside FP4.","If consistency is the bottleneck, learned or adaptive 2D blocks and fused scale metadata could push past the 32×32 efficiency–accuracy trade-off without new number formats.","Selective higher precision may generalize: other noise-amplifying ops (softmax, layer-norm, router logits in MoE) could be the next places to keep above FP4 rather than whole layers."],"forward_implications":["Stable FP4 training at LLM scale can be reached mainly by making quantization transpose-invariant, not only by shrinking blocks or raising scale precision.","Most transformer linear layers can stay FP4; only Q/K need higher precision (MXFP8) in this design.","Weight memory and ideal linear throughput can improve substantially (paper estimates ~65% weight memory cut and up to ~3.6× linear throughput on OLMo-1B) once native kernels exist.","Larger models can close the FP4–BF16 gap further once systematic scale bias is removed.","Hardware and formats should expose square-block or otherwise transpose-consistent scaling paths, not only 1D microscaling."],"fun_headline_variants":["2D block FP4 keeps scales matched across transpose for stable LLM training","Transposition-invariant FP4 quantization closes most of the gap to BF16","Scale mismatch from 1D blocks destabilizes FP4; 2D blocks fix it","Forward-backward scale consistency enables end-to-end FP4 at 30B","Square-block FP4 plus MXFP8 for Q/K trains stably up to 100B tokens"],"cache_read_input_tokens":128,"weakest_assumption_plain":"That software-emulated FP4 microscaling on hardware without native FP4 tensor cores behaves like real MXFP4/NVFP4 hardware for the stability conclusions that matter.","fun_headline_variants_meta":{"raw":{"variants":["2D block FP4 keeps scales matched across transpose for stable LLM training","Transposition-invariant FP4 quantization closes most of the gap to BF16","Scale mismatch from 1D blocks destabilizes FP4; 2D blocks fix it","Forward-backward scale consistency enables end-to-end FP4 at 30B","Square-block FP4 plus MXFP8 for Q/K trains stably up to 100B tokens"]},"model":"grok-4.5","effort":"low","cost_usd":0.002347,"raw_usage":{"total_tokens":1031,"prompt_tokens":851,"num_sources_used":0,"completion_tokens":113,"cost_in_usd_ticks":23468000,"prompt_tokens_details":{"text_tokens":851,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":67,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":851,"tokens_out":113,"duration_ms":3270,"temperature":1.0,"reasoning_tokens":67,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T05:00:12.874736+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train the same OLMo-1B/7B or 30B MoE recipe on native FP4 tensor-core hardware (or a verified bit-accurate emulator) with 1D vs 2D blocks only, and check whether 1D still shows the forward–backward scale mismatch and early divergence while 2D stays within ~1% of BF16 loss and downstream accuracy through 100B tokens.","supporting_citations":[],"review_version":1}