{"id":"a6531b0e-390a-4513-83a3-4ba36be5ba18","arxiv_id":"2412.08832","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A Tensor Core based Fast Walsh-Hadamard Transform kernel that runs 1.1-1.4x faster than the existing CUDA library, with up to 3.5x peak speedup.","lead":"HadaCore is a faster way to apply a mathematical rotation, called a Hadamard transform, on modern Nvidia GPUs by using special matrix hardware. It speeds up a step used in quantized AI models by 10 to 40 percent, with larger gains in some cases.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline speedup numbers compare HadaCore against an unmodified baseline, while Appendix B shows a one-line in-place patch makes that baseline substantially faster; re-benchmarking against the patched baseline may erase much of the reported 8M/16M gains.","rationale":"The reader's weakest assumption identifies exactly the concern I would flag: the comparison baseline for the central speedup claim is not the strongest legitimate baseline. The paper itself provides evidence in Appendix B that the baseline library can be made faster with a one-line in-place change, and Section 4.1 acknowledges that HadaCore's gains at 8M/16M elements come from the same in-place optimization. For a claim that HadaCore is faster than the state-of-the-art implementation of the original algorithm, the baseline must be allowed the same memory-layout optimization. If the patched baseline is used, the reported average and peak speedups will likely shrink, though the algorithm may still retain some advantage. The kernel's correctness is supported by unit tests against explicit Hadamard multiplication, and the MMLU end-to-end test is a real validation of numerical utility, so this is not a rejection of the method. It is a request for an apples-to-apples performance measurement, plus repetitions and commit-level code to make the measurement reproducible. Since the reader already conditioned acceptance on re-benchmarking against the improved baseline and reporting repetitions, my stress-test does not change the reader's verdict: the conditional remains the right call.","tokens_in":7496,"tokens_out":5659,"duration_ms":61173,"concrete_test":"Rebenchmark on the same A100 and H100 hardware using the Appendix B patch as the baseline (`at::Tensor out = x;` instead of `torch::empty_like(x)`), with HadaCore also run in the same in-place mode. Report mean and minimum runtimes over at least 10 repetitions per (Hadamard size, element count) cell, and recompute the average and peak speedups. If the patched-baseline speedup at 8M/16M elements is materially smaller than the unpatched-baseline speedup, or if the average speedup falls below 1.0 in any size family, then the headline speedup range should be revised downward and the conditional acceptance should require publishing patched-baseline comparisons.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing issue is not the recursive Tensor-Core construction itself, which appears internally consistent, but the comparison baseline for the headline speedups. Section 4.1 attributes part of HadaCore's 8M/16M-element gains to an in-place rotation optimization, and Appendix B shows that the Dao AI Lab kernel can receive the same optimization by changing `at::Tensor out = torch::empty_like(x)` to `at::Tensor out = x` in `/csrc/fast_hadamard_transform.cpp`. Because the paper's main tables (Figures 6/7 and 10/11) compare against the unmodified kernel, the baseline pays for an output allocation and for source/destination L2 cache eviction at tensor sizes where HadaCore is measured in-place. The reported 1.1-1.4x (A100) and 1.0-1.3x (H100) averages, and especially the 3.5x/3.6x peaks, are therefore not cleanly attributable to HadaCore's Tensor-Core design. The central claim would be fully supported only if HadaCore also beats the baseline after the same one-line in-place change, or if the paper separately reports the algorithmic speedup and the memory-layout speedup. This is a measurable, fixable issue rather than a flaw in the kernel's mathematical construction.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"HadaCore is a CUDA kernel that computes the Fast Walsh-Hadamard Transform using 16x16 Tensor Core matrix multiply instructions as the base case, with a recursive decomposition that handles sizes up to 2^15 and non-power-of-16 sizes. The paper reports runtime speedups over the Dao AI Lab fast-hadamard-transform library on A100 and H100 GPUs for FP16 and BF16, and reports an MMLU experiment on Llama-3.1 8B with FP8 attention showing that Hadamard rotations implemented by HadaCore give accuracy close to the Dao AI Lab kernel. The authors provide source code and unit tests.","tokens_in":7778,"tokens_out":6823,"duration_ms":70267,"significance":"The paper addresses a real practical bottleneck: Hadamard rotations in QuaRot/SpinQuant-style quantized LLM inference are often implemented as a dedicated FWHT kernel, and any speedup there reduces end-to-end overhead. The central construction is internally coherent: replacing the size-2 butterfly base case by a 16x16 Tensor Core mma increases arithmetic by about 2x while exposing roughly 8x higher FLOPS, and the claimed reduction in synchronization and data shuffling is plausible. The manuscript has real strengths: parameter-free complexity analysis, unit tests against explicit Hadamard multiplication, and public code. However, the headline speedup claim currently rests on a baseline that Appendix B itself shows can be improved by a one-line in-place change, and the MMLU comparison has no uncertainty quantification. These are fixable benchmarking issues rather than flaws in the algorithm.","major_comments":[{"comment":"The reported speedups compare HadaCore against the unmodified Dao AI Lab kernel. Appendix B shows that replacing `at::Tensor out = torch::empty_like(x)` with `at::Tensor out = x` in `/csrc/fast_hadamard_transform.cpp` substantially improves the baseline's performance, and Section 4.1 attributes part of HadaCore's 8M/16M-element gains to in-place rotations. Because the main tables and figures use the unmodified kernel, the baseline at large tensor sizes pays for an output allocation and for source/destination L2 cache eviction that HadaCore avoids. The 1.1-1.4x (A100) and 1.0-1.3x (H100) averages, and especially the 3.5x/3.6x peaks, are therefore not cleanly attributable to the Tensor-Core design. The authors should rerun the full comparison against the in-place-patched baseline, or separately report an algorithmic speedup (same memory behavior) and a memory-layout speedup. If the patched baseline closes most of the gap, the central claim must be reframed.","section":"Section 4.1 and Appendix B, Figures 6/7/10/11"},{"comment":"The paper reports single point estimates of 65.45 (Dao AI Lab kernel) and 65.09 (HadaCore) for FP8 attention with rotation, compared to 64.40 without rotation and 65.38 FP16 baseline. No error bars, number of seeds, or statistical comparison is given. The 0.36-point gap between HadaCore and the Dao kernel, and the 0.29-point gap to the FP16 baseline, are within typical 5-shot MMLU run-to-run variance, so the claim that HadaCore 'maintains comparable quantization error reduction' is not yet supported. Please report multiple runs with standard deviations or confidence intervals, and ideally the same random subsets for a paired comparison.","section":"Section 4.2 (MMLU table)"}],"minor_comments":[{"comment":"The sentence 'The fast-hadamard-transform library now outperforms memcpy (which they used as a lower bound in their benchmarks) in some cases' is surprising if memcpy was presented as a lower bound; clarify why a transform can be faster than a copy, or clarify what lower-bound claim is being made.","section":"Appendix B"},{"comment":"The tables and figures would benefit from a statement of measurement methodology (number of repetitions, warm-up, GPU clock locking, and run-to-run variance). At present the reader cannot distinguish real speedups from noise at the 1.0-1.1x level.","section":"Appendix A, Figures 6/7/10/11"},{"comment":"The MMLU table is not numbered or captioned, and the evaluation setup (number of MMLU subjects, exact prompt template, batch size, whether the same random examples are used) is not described; please add these details.","section":"Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is honest in disclosing the in-place optimization in Appendix B, but the authors have not yet integrated that disclosure into the headline comparisons. I suspect the Tensor-Core construction itself is sound; the main risk is that the reported speedups are dominated by the memory-layout change. Re-benchmarking against the patched baseline is a straightforward fix, so I recommend major revision rather than rejection. No concern about scope or novelty."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: the tensor-core base-case for the FWHT is a genuine new kernel idea, and the paper is transparent about its own caveats. But the headline speedups compare against a baseline that the authors themselves show can be improved with a one-line in-place change, so treat the average 1.1–1.4x and peak 3.5x numbers with caution until you see a re-benchmark against the patched baseline.\n\nWhat's actually new: mapping the 16x16 Hadamard transform to two Tensor Core mma operations, using a transpose-and-repeat for sizes above 256, and warp shuffles for coalesced transposes. The complexity trade-off (2x flops vs 8x FLOPS) is coherent, and the unit tests against explicit Hadamard multiplication support correctness. Releasing code is a plus. The paper also honestly documents that part of the 8M/16M-element gain comes from an in-place rotation optimization, and Appendix B shows exactly how the baseline could be patched the same way. That is more than most kernel papers do.\n\nThe soft spot is the main comparison. The figures compare against the unmodified Dao AI Lab kernel, which allocates a separate output tensor. At 8M–16M elements, that allocation and the resulting L2 eviction cost are exactly the issue the in-place optimization fixes. So the headline numbers mix an algorithmic speedup with a memory-layout speedup. The authors do not hide this, but they don't separate the two. A re-run against the in-place baseline is needed; my guess is the average gains drop to maybe 1.0–1.2x, with peaks still present at particular sizes.\n\nMinor issues: the MMLU result is a single run with no error bars, but it's a secondary point. The H100 numbers are weaker and the authors attribute that to pre-Hopper focus—fair, but it means the \"tensor-core acceleration\" claim is strongest on A100.\n\nWho this is for: people optimizing Hadamard rotations in quantized LLM inference, and kernel engineers interested in mapping butterfly-style transforms to tensor cores. It deserves a serious referee—the technique is novel, the writeup is honest, and the benchmarking flaw is fixable. I'd send it out.\n\nRecommendation: engage with it, but ask for a re-benchmark against the in-place baseline and separate the algorithmic and memory-layout contributions. Also ask for the code version or commit hash.","headline":"Genuine tensor-core FWHT kernel, but headline speedups overstate the gain because the baseline lacks the in-place optimization the authors themselves identify.","tokens_in":8294,"tokens_out":2756,"would_cite":false,"duration_ms":26347,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HadaCore turns the Fast Walsh-Hadamard Transform into Tensor Core matmuls and runs up to 3.6x faster while preserving FP8-attention accuracy.","keywords":["Hadamard transform","Tensor Cores","CUDA kernel","LLM quantization","FP8 attention","inference acceleration","Walsh-Hadamard","in-place rotation"],"falsifier":"Apply the appendix's one-line change (write the Hadamard result into the input tensor instead of a freshly allocated output) to the baseline kernel, then rerun the 8M- and 16M-element FP16 benchmarks on A100 and H100; if HadaCore's speedup over this matched baseline is near 1.0 at those sizes, the largest reported gains come from cache behavior rather than the Tensor Core base case.","tokens_in":7332,"feed_emoji":"⚡","tokens_out":10808,"duration_ms":103172,"temperature":0.7,"pith_summary":"HadaCore is a CUDA kernel that rewrites the Fast Walsh-Hadamard Transform so that the base case is a 16-by-16 matrix multiply executed on GPU Tensor Cores rather than the usual pairwise butterfly. The paper claims this hardware-aware decomposition keeps the same asymptotic cost while cutting synchronization and data movement enough to run 1.1-1.4x faster on average on A100 and 1.0-1.3x faster on H100, with peak gains of 3.5x and 3.6x, against the existing optimized CUDA implementation. It also reports that the faster kernel does not degrade quantization quality: on Llama-3.1 8B with FP8 attention, HadaCore reaches 65.09 average MMLU accuracy versus 65.45 for the existing kernel and 64.40 with no rotation. The practical stakes are that Hadamard rotations are the standard cheap way to remove activation outliers before low-precision LLM inference, so a faster drop-in transform lowers the overhead of QuaRot- and SpinQuant-style pipelines. The paper additionally identifies an in-place rotation trick that reduces L2-cache pressure for large tensors and attributes part of the largest-size gains to it.","feed_headline":"Fast Hadamard transform runs up to 3.6x faster on Tensor Cores","feed_subtitle":"A 16x16 Tensor Core base case speeds Hadamard rotations in quantized LLM inference while keeping FP8 accuracy.","key_machinery":"The central mechanism is the 16-by-16 Hadamard multiply as the base case, executed with the Tensor Core `mma` instruction: two 16x16-by-16x8 matrix operations combine into one 16x16-by-16x16 multiply, and the fragment lives in registers. The governing identity is the Kronecker structure of Sylvester's Hadamard construction: a 2n-size transform can be obtained by applying an n-size transform, permuting elements, and applying it again, which in this kernel is realized by transposing a 16-by-16 chunk so that two 16-size base-case applications separated by the transpose produce a 256-size transform. Non-power-of-16 sizes are handled by tiling a smaller Hadamard along the diagonal of the final 16-by-16 matrix. This machinery does the work of turning four butterfly levels of the textbook algorithm into one Tensor Core matmul, which is why the reported speedups come from reduced synchronization and data exchange rather than reduced arithmetic.","core_discovery":"The discovery is that the FWHT's recursive stages can be reorganized so that the base case is one 16-by-16 Walsh-Hadamard multiply executed by two Tensor Core `mma` operations, with the data held in registers. Applying that base case to 16-by-16 chunks, transposing, and applying it again produces a 256-element transform, and larger sizes are reached by chunking, a threadblock-level transpose, and another round of base cases. The paper shows this reorganization costs at least twice as many floating-point operations as the textbook algorithm ($4mn\\log_2(n)$ versus $2mn\\log_2(n)$) yet is faster on A100 and H100 because Tensor Cores give roughly 8x the throughput and because the kernel needs fewer synchronization barriers and less data shuffling. The same kernel in FP16 and BF16 stays numerically accurate, and an end-to-end Llama-3.1 8B run with FP8 attention gives 65.09 average MMLU accuracy with HadaCore rotations versus 65.45 with the baseline kernel and 64.40 without rotations. The claim is thus a better hardware mapping of the same algorithm, not a cheaper algorithm.","pith_inferences":["Editorial inference: the paper's own appendix implies that a fair head-to-head at large element counts should use the in-place variant of the baseline; with that one-line change the reported average speedup at 8M and 16M elements is likely to shrink, though the Tensor Core advantage at other sizes may remain.","Editorial inference: the 16-by-16 base-case mapping should transfer to other architectures with matrix-multiply-accumulate hardware and to other Kronecker-structured orthogonal transforms, since nothing in the argument is specific to Nvidia's `mma` instruction except the register layout.","Editorial inference: kernel fusion is the natural next step, because HadaCore already leaves data in registers and uses FP32 accumulation for BF16, so a fused Hadamard-plus-quantization kernel could avoid the global-memory round trip identified as future work."],"forward_implications":["HadaCore can replace the Hadamard rotation step in existing quantized-inference pipelines without retraining or changing the quantization scheme, because the MMLU run shows rotated FP8 attention accuracy is preserved.","The largest element counts (8M on A100, 16M on H100) benefit most, which the paper attributes to in-place rotation keeping source and destination within L2 cache instead of evicting each other.","BF16 users get the same speedup pattern as FP16, with only a small overhead from FP32 accumulation and conversion.","Hadamard sizes larger than 256 avoid cross-threadblock synchronization except for one shared-memory transpose, keeping the cost of the serial log-scale recursion small."],"supporting_citations":[{"why":"The GPU instruction-set reference that documents the Tensor Core matrix-multiply instructions used as the base case.","marker":"[2]"},{"why":"The established CUDA Fast Walsh-Hadamard kernel whose runtimes are the baseline for every speedup in the paper.","marker":"[3]"},{"why":"The classical paper defining the recursive Fast Walsh-Hadamard Transform that HadaCore reworks.","marker":"[4]"},{"why":"The benchmark suite used to verify that the accelerated kernel preserves FP8-attention accuracy on Llama-3.1 8B.","marker":"[5]"}],"fun_headline_variants":["HadaCore: Hadamard transform on Tensor Cores up to 3.6x faster","Tensor Core base case speeds Hadamard transform 3.6x, keeps FP8 accuracy","Hadamard transform with 2x FLOPs still 3.6x faster on Tensor Cores","Reorganized Hadamard kernel gives up to 3.6x speedup on A100 and H100","Hadamard transform on Tensor Cores: 3.6x peak speedup, LLM accuracy intact"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported speedups assume the published baseline kernel is the correct comparison; the appendix shows the baseline can be made substantially faster with a one-line in-place-output change, and part of HadaCore's biggest gains is attributed to that same change.","fun_headline_variants_meta":{"raw":{"variants":["HadaCore: Hadamard transform on Tensor Cores up to 3.6x faster","Tensor Core base case speeds Hadamard transform 3.6x, keeps FP8 accuracy","Hadamard transform with 2x FLOPs still 3.6x faster on Tensor Cores","Reorganized Hadamard kernel gives up to 3.6x speedup on A100 and H100","Hadamard transform on Tensor Cores: 3.6x peak speedup, LLM accuracy intact"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000348,"raw_usage":{"total_tokens":1912,"prompt_tokens":961,"completion_tokens":951,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":577,"completion_tokens_details":{"reasoning_tokens":820}},"tokens_in":577,"tokens_out":951,"duration_ms":8794,"temperature":1.0,"reasoning_tokens":820,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:29:39.317843+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Apply the appendix's one-line change (write the Hadamard result into the input tensor instead of a freshly allocated output) to the baseline kernel, then rerun the 8M- and 16M-element FP16 benchmarks on A100 and H100; if HadaCore's speedup over this matched baseline is near 1.0 at those sizes, the largest reported gains come from cache behavior rather than the Tensor Core base case.","supporting_citations":[{"cited_title":"CUDA C++ Programming Guide","cited_arxiv_id":null,"evidence_quote":"The GPU instruction-set reference that documents the Tensor Core matrix-multiply instructions used as the base case."},{"cited_title":"Fast Hadamard Transform in CUDA, with a PyTorch interface","cited_arxiv_id":null,"evidence_quote":"The established CUDA Fast Walsh-Hadamard kernel whose runtimes are the baseline for every speedup in the paper."},{"cited_title":"Unified Matrix Treatment of the Fast Walsh-Hadamard Transform","cited_arxiv_id":null,"evidence_quote":"The classical paper defining the recursive Fast Walsh-Hadamard Transform that HadaCore reworks."}],"review_version":1}