{"id":"5d960901-945e-4082-bc68-1389e0dbfb9c","arxiv_id":"2607.21446","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Using learned 32×32 Kronecker block transforms as an online activation smoother improves W4A4 image quality of PixArt-Sigma, SANA, and FLUX.1-schnell over SVDQuant and LoRaQ, with a kernel up to 14% faster than SmoothQuant.","lead":"KroQuant is a post-training quantization method for diffusion image generators that mixes each 32-long chunk of activations with a learned 32×32 matrix built from tiny 2×2 building blocks, then fixes residual weight errors with an existing low-rank correction. In tests on three text-to-image models at 4-bit precision, it usually gets closer to full-precision output than prior quantizers, and on one GPU configuration its kernel is up to 14% faster than the standard SmoothQuant","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central quality claim rests on the untested assumption that 32-element blocks capture all activation outliers; the paper's own FLUX results show this assumption fails in double-stream architectures.","rationale":"The reader's weakest_assumption precisely identifies the 32-block locality as the load-bearing constraint, and the paper's own FLUX/MJHQ numbers are consistent with that assumption being only partially satisfied. The reader's verdict CONDITIONAL is appropriate: the method has a clear empirical gain on PixArt-Σ and SANA, but the central claim of 'closer to FP reference' is not universal, and the failure mode is not analyzed. The concrete permutation test would settle whether the block boundary is the cause, which is directly relevant to whether the method generalizes beyond the favorable single-stream architectures. I considered raising the K1/K2 kernel-speed/storage conflation, but the paper's body is transparent about the two variants and the central speed claim is supported for K1; the block-locality issue is more fundamental because it threatens the quality claim itself. The verdict remains CONDITIONAL: the method is promising, but the paper should either add the permutation-stability analysis or explicitly scope the claims to architectures where the 32-block assumption holds, and release code/data to verify the numbers.","tokens_in":15231,"tokens_out":17454,"duration_ms":164339,"concrete_test":"On FLUX.1-schnell, apply a fixed random permutation π to the hidden-channel dimension of all activations and the corresponding weight columns before learning T (i.e., quantize A·P^T and W·P^T, with T learned on the permuted channels). Run the same calibration and MJHQ-30K evaluation for several permutations. If LPIPS/IR vary significantly across permutations, then the method's quality depends on the arbitrary channel ordering, confirming the 32-block boundary is the active constraint. If results are permutation-stable, the FLUX deficit is due to something else (e.g., token heterogeneity) and the block-locality assumption is not the bottleneck.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's core assumption (Section 3.2, Eq. 4) is that activation outliers can be suppressed by mixing only within consecutive 32-channel blocks, matching the MXFP4e2 quantizer's block size. The learned T is block-diagonal with 32×32 blocks, so it cannot mix channels across block boundaries. If outlier directions span multiple blocks, or if text and image tokens in FLUX double-stream layers require different rotations in the same block, the transform is structurally incapable of fixing the quantization error. The paper's own Table 1 provides direct evidence: on FLUX.1-schnell / MJHQ-30K, KroQuant's IR (0.880) is far below SVDQuant (0.952) and its LPIPS (0.336) is slightly worse than LoRaQ (0.331), despite large gains on SDCI. The paper attributes this to 'double-stream heterogeneity' but offers no diagnostic to confirm that the 32-block boundary, rather than some other limitation, is the cause. The Limitations section admits the Kronecker parametrization grows too slowly with block size and that joint optimization is unstable, but never tests the fundamental 32-block boundary. Because the central claim promises outputs closer to FP16 than baselines on most configurations, this assumption is load-bearing: if it fails on a given architecture, the method degrades toward or below the SmoothQuant baseline.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes KroQuant, a post-training quantization method for diffusion transformers that inserts a learned Kronecker-structured block-diagonal transform into the activation path before W4A4 MXFP4 quantization. Each 32×32 block is parameterized as a Kronecker product of five 2×2 unit-determinant LU factors, giving 15 learnable parameters per block, with the Hadamard transform as initialization. The transform is applied online (because AdaLN prevents offline absorption), and the authors provide fused Triton kernels (K1 dense, K2 on-the-fly) claiming speed comparable to or better than per-channel scaling. After transform calibration, LoRaQ weight calibration is applied. Experiments on PixArt-Σ, SANA, and FLUX.1-schnell report FID, IR, LPIPS, and PSNR for MJHQ-30K and SDCI, plus kernel-latency benchmarks on an MI350. The paper also includes an ablation, kernel pseudocode, a normalization-commutativity study, and a limitations section.","tokens_in":15552,"tokens_out":4561,"duration_ms":44959,"significance":"If the results hold, KroQuant is a useful contribution to the DiT PTQ toolbox: it is, to my knowledge, the first learned block-diagonal activation transform specifically matched to MXFP4's 32-wide blocks, and it is shown to improve over both a fixed Hadamard at the same block size and over per-channel scaling within a LoRaQ-style pipeline. The parameter-count argument and invertibility construction are clean and clearly stated. The kernel measurements provide credible evidence that the online cost is competitive with per-channel scaling on the tested MI350 shapes. The authors are also candid about limitations: FLUX results are mixed, the Kronecker parameterization grows too slowly beyond n=32, and joint optimization is unstable. However, the abstract's blanket claim of superiority over SVDQuant and LoRaQ is contradicted by the paper's own Table 1 on FLUX.1-schnell/MJHQ-30K, and the central 32-block assumption is not directly tested. The contribution is therefore significant but its scope must be stated more carefully.","major_comments":[{"comment":"The abstract states that KroQuant 'produces outputs closer to the FP reference than SVDQuant and LoRaQ on MJHQ-30K and SDCI.' Table 1 contradicts this on FLUX.1-schnell/MJHQ-30K: LPIPS is 0.336 versus 0.331 for LoRaQ and IR is 0.880 versus 0.952 for SVDQuant and 0.921 for LoRaQ. While KroQuant has the best FID and PSNR there, the claim of being 'closer to FP' is not metric-wise true. The main text acknowledges this as 'mixed,' but the abstract does not. This overreach should be corrected, and the conclusions should be scoped to the configurations and metrics actually supporting the claim.","section":"Abstract and §4.2, Table 1"},{"comment":"The load-bearing design assumption is that consecutive 32-element blocks can capture all relevant activation outlier structure. The transform T is block-diagonal with 32×32 blocks and cannot mix across block boundaries. The FLUX MJHQ results are consistent with this assumption failing in double-stream layers, but the paper does not provide a direct test. No experiment varies the block size, permutes channel grouping, or measures how much quantization-error reduction would be obtained by allowing cross-block coupling. Since the central claim depends on the 32-block structure being sufficient, the authors should either add such diagnostics or explicitly reframe the method as targeting architectures where this assumption holds (e.g., the single-stream PixArt/SANA settings).","section":"§3.2, Eq. (4), and §4.2"},{"comment":"The limitations note that the Kronecker construction has only 3 log2 n parameters per n×n block and 'grows too slowly with block size,' so the method is tied to n=32. This is more than a future-work remark: it means the paper does not actually deliver a general solution to the 'need' articulated in Section 1 for a transform 'more expressive than a fixed Hadamard at the same block size' — at n=32 it is more expressive, but the architecture cannot scale to the larger block sizes that would test the boundary assumption. The authors should acknowledge this directly in the main text and avoid implying that the method generalizes beyond the n=32 setting.","section":"§5, Limitations and §3.2"}],"minor_comments":[{"comment":"The speedup ratio notation 'Scale/K1 Speedup' is confusing. The caption explains that 'Method B is X faster than Method A,' but the column header suggests the opposite. Please rename columns to e.g. 'K1 vs Scale' and state the convention near the table.","section":"Table 3 and Table 4"},{"comment":"The parametrization G_k = L_k U_k with U_k = [[a_k, b_k],[0, -1/a_k]] has det G_k = -1 for every k. Since the product of five factors yields det K = -1, all K blocks are in the determinant -1 component of GL(32). This is not a problem for the method, but it may be worth stating explicitly to avoid confusion with unit-determinant 'for every factor' language.","section":"§3.2, Eq. (6)"},{"comment":"The calibration and evaluation details are mostly present, but the evaluation protocol for MJHQ-30K and SDCI could say explicitly how many seeds are used and whether the same 5,000 images are used for all methods. The phrase 'following the evaluation split used in prior work' is a bit vague.","section":"§4.1"},{"comment":"Table 4 reports a 0.99× Scale/K1 ratio at (m=4096, d=12288), while the text says 'matches or improves on per-channel scaling on every shape except the largest.' The exception is not highlighted in the table; please add a footnote or marker.","section":"Appendix A"},{"comment":"There are a few typos and inconsistent spellings: 'KroQuant' vs 'Kroquant' in Appendix D.2, 'Pixart-Σ' vs 'PixArt-Σ,' and 'sDCI' in the assets list. Please proofread newly introduced names.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for a machine-learning systems/quantization journal and the mathematical core is clean. The main issue is that the abstract and general conclusions overstate the experimental support, particularly on FLUX/MJHQ. The stress-test concern about the 32-block assumption is real but can be addressed by scope restriction or additional diagnostics; it is not a fundamental invalidation. I would be comfortable with acceptance after the claims are aligned with Table 1 and the block-size limitation is treated as a scoping condition rather than an aside."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: this is a genuinely new transform family for DiT PTQ — learned Kronecker-product 32×32 blocks with 15 parameters, unit-determinant LU factors, Hadamard init — and the math is clean. The ablation is convincing: with LoRaQ held fixed, the learned block beats a fixed Hadamard-32 by 0.035 LPIPS and 0.8 dB PSNR on PixArt-Σ, which is exactly the right controlled comparison. The kernel work is also a real contribution: K1 matches or beats per-channel scaling on most shapes, and the K2 rematerialization idea is clever even if it's only marginally cheaper on some shapes.\n\nThe paper does not deserve a desk reject. It deserves a referee. But the top-line claim in the abstract is not what Table 1 shows. On FLUX/MJHQ, KroQuant's IR (0.880) is far below SVDQuant (0.952) and its LPIPS (0.336) is slightly worse than LoRaQ (0.331). The abstract says \"closer than SVDQuant and LoRaQ on MJHQ-30K and SDCI,\" which only holds on SDCI for that model. That is a real accuracy problem in the communication of the result, not a nitpick.\n\nThe stress-test concern about the 32-block boundary is fair, and I think it lands. The entire method assumes outliers can be suppressed by mixing within consecutive 32-channel groups, matching MXFP4e2's block size. The FLUX result is consistent with that assumption failing when text and image tokens with different distributions share a block. The paper acknowledges double-stream heterogeneity, but offers no diagnostic — e.g., a head-to-head with a larger block transform, or a per-block outlier statistic — to confirm the mechanism. Given that the method is structurally wedded to n=32 (the Limitations section admits the parameterization grows too slowly with block size), this is the load-bearing assumption and it remains untested.\n\nOther concerns are more modest. No code or data are shipped, which for an empirical PTQ paper is a real gap — the kernel speedups are hard to verify without it, and there are no error bars or repeated-seed statistics anywhere. The self-citation to LoRaQ is not a problem: the comparison holds LoRaQ fixed and varies only the transform, and the authors are transparent about the dependency.\n\nWho is this for? Researchers and practitioners working on 4-bit diffusion transformer inference. It is a solid, deployable step if the numbers hold up. I would send this to review, but require (a) corrected abstract, (b) a diagnostic or honest framing of the block-boundary assumption, and (c) code or at least detailed kernel baselines. With those, it's a useful contribution.","headline":"Learned Kronecker block transforms for DiT PTQ are a genuinely new and plausible idea with clean math and a fair controlled ablation, but the abstract oversells the results and the 32-block assumption is load-bearing and untested.","tokens_in":16118,"tokens_out":1991,"would_cite":true,"duration_ms":19716,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"KroQuant claims that a learned 32×32 Kronecker block transform, run online before quantizing each linear layer, closes most of the quality gap between W4A4 and full-precision diffusion transformers while adding no inference slowdown.","keywords":["post-training quantization","diffusion transformers","activation outliers","Kronecker product","block-diagonal transform","4-bit quantization","MXFP4","efficient inference"],"falsifier":"Run KroQuant's calibration on a synthetic linear layer whose activation distribution has two large outlier channels separated by more than 32 indices, so no single 32×32 block can mix them, then quantize at W4A4; the method should fail to remove those outliers and the output error should be close to a per-channel-scaling baseline. Alternatively, take a calibrated DiT layer, randomly permute the channel dimension of the learned transform, and re-measure LPIPS or output MSE: if the observed gains over SmoothQuant largely disappear, the block-local alignment — not the learned rotation per se — is","tokens_in":15053,"feed_emoji":"🧮","tokens_out":10512,"duration_ms":94195,"temperature":0.7,"pith_summary":"The paper is trying to solve a specific bottleneck: when diffusion transformers are quantized to 4-bit weights and activations, a few outlier channels dominate the 4-bit dynamic range and ruin output quality. The usual remedy is to rotate or rescale activations before quantizing and fold the inverse into the weights, but in diffusion transformers the normalization layers between blocks make this impossible to do offline, so the transform must run online at every denoising step. KroQuant's claim is that the right transform for this setting is a learned block-diagonal matrix with 32×32 blocks, parameterized as a Kronecker product of five 2×2 unit-determinant factors and initialized at the Hadamard transform. This gives more expressivity than a fixed Hadamard at the same block size and fewer parameters than per-channel scaling, while applying as a single tensor-core GEMM per block. Combined with LoRaQ's offline weight calibration, the paper reports outputs closer to the FP16 reference than SVDQuant and LoRaQ-with-SmoothQuant on PixArt-Σ, SANA, and most of FLUX.1-schnell at W4A4, with a quantizer kernel up to 14% faster than per-channel scaling.","feed_headline":"Learned 32x32 block transform closes 4-bit quantization gap","feed_subtitle":"On PixArt-Σ and SANA, it beats both baselines and runs as fast as per-channel scaling.","key_machinery":"The load-bearing object is the Kronecker-structured 32×32 invertible block transform T, block-diagonal with blocks K^(j) = G1⊗G2⊗G3⊗G4⊗G5, where each G_k is a 2×2 unit-determinant LU matrix with three scalars. It mixes channels inside each 32-element block before the MX quantizer, spreading outlier magnitudes so the 4-bit format is used more evenly; it is learned per layer by minimizing quantized output MSE with auxiliary per-operand regularization, initialized at the Hadamard transform, and applied online as small tensor-core GEMMs, with its inverse folded into weights offline. The two kernel variants — precomputed dense blocks (K1) and on-the-fly Kronecker reconstruction from 15 scalars (K","core_discovery":"On the paper's own terms, the central discovery is that the expressivity-versus-cost trade-off for online activation transforms in diffusion-transformer quantization has a sweet spot: a learned, block-diagonal transform whose 32×32 blocks align exactly with the block width of the MXFP4e2 quantizer. Each block is built as a Kronecker product of five 2×2 factors, each factor parameterized by three scalars through a unit-determinant LU form, so a full 32×32 block costs 15 learnable parameters. Since the Hadamard matrix is itself a Kronecker product of five identical 2×2 factors, the transform can be initialized at a block-Hadamard rotation and then tuned per layer with a straight-through estima","pith_inferences":["Editorial inference: because the block size is tied to the MX quantizer's 32-wide blocks, the same construction should transfer to other blockwise formats (e.g., 16-element blocks) by changing the number of Kronecker factors, suggesting a general recipe for learned rotation at quantizer granularity.","Editorial inference: the 15-parameter-per-block parametrization sits between diagonal scaling (32 scalars per block) and a dense rotation (1024 scalars), so one could tune expressivity per layer by choosing how many Kronecker factors to learn; the paper's fixed five-factor choice is a conservative default.","Editorial inference: a direct stress test of the paper's core assumption is to permute channel indices after calibration before applying the learned transform; if quality drops toward the SmoothQuant baseline, the gains depend on outlier correlations coinciding with 32-channel blocks rather than on the learned rotation per se."],"forward_implications":["At W4A4 with the MXFP4e2 format, KroQuant-with-LoRaQ beats SVDQuant and LoRaQ-with-SmoothQuant on LPIPS and PSNR for PixArt-Σ and SANA on both MJHQ-30K and SDCI, and closes most of the FID gap to the FP16 reference.","The learned transform strictly improves on its fixed Hadamard initialization: holding LoRaQ fixed, the 15 learnable parameters per block cut LPIPS by 0.035 and raise PSNR by 0.8 dB on PixArt-Σ.","The kernel cost is not a barrier: K1 matches or beats per-channel scaling on every benchmarked shape (1.00–1.14×) and K2 is 1.30–2.45× faster than a fast Walsh–Hadamard transform, while handling hidden dimensions divisible by 32 without padding.","The transform is a drop-in replacement for the SmoothQuant front-end in weight-splitting PTQ pipelines, so the LoRaQ weight correction can be reused unchanged.","On FLUX.1's double-stream architecture the gain is partial (best FID/PSNR but not IR/LPIPS on MJHQ-30K), which the paper attributes to joint attention mixing text and image tokens with different activation distributions."],"fun_headline_variants":["Kronecker block transform trims 4-bit diffusion quantization gap","15-param block transform beats larger PTQ baselines at W4A4","Faster than SmoothQuant, tighter 4-bit DiT quantization","KroQuant: lean Kronecker transform for accurate W4A4 diffusion","KroQuant beats SVDQuant & LoRaQ on MJHQ/SDCI"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that activation outliers in DiT linear layers can be neutralized by mixing only within independent 32-element blocks aligned with the MX quantizer; if outlier structure spans across blocks or involves long-range channel correlations, KroQuant cannot suppress it and its quality advantage shrinks toward per-channel scaling.","fun_headline_variants_meta":{"raw":{"variants":["Kronecker block transform trims 4-bit diffusion quantization gap","15-param block transform beats larger PTQ baselines at W4A4","Faster than SmoothQuant, tighter 4-bit DiT quantization","KroQuant: lean Kronecker transform for accurate W4A4 diffusion","KroQuant beats SVDQuant & LoRaQ on MJHQ/SDCI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001787,"raw_usage":{"total_tokens":6943,"prompt_tokens":873,"completion_tokens":6070,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":617,"completion_tokens_details":{"reasoning_tokens":5969}},"tokens_in":617,"tokens_out":6070,"duration_ms":40744,"temperature":1.0,"reasoning_tokens":5969,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T07:25:29.419101+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run KroQuant's calibration on a synthetic linear layer whose activation distribution has two large outlier channels separated by more than 32 indices, so no single 32×32 block can mix them, then quantize at W4A4; the method should fail to remove those outliers and the output error should be close to a per-channel-scaling baseline. Alternatively, take a calibrated DiT layer, randomly permute the channel dimension of the learned transform, and re-measure LPIPS or output MSE: if the observed gains over SmoothQuant largely disappear, the block-local alignment — not the learned rotation per se — is","supporting_citations":[],"review_version":1}