{"id":"73d0cf7c-1c21-4784-a4b5-a9ad24a732d5","arxiv_id":"2509.01229","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A W4A8 GEMM kernel using shift-based overflow-safe dequantization and a hardware-scheduled pipeline reports up to 2.9x speedup over prior W4A8 kernels.","lead":"LiquidGEMM speeds up 4-bit weight, 8-bit activation matrix operations on NVIDIA GPUs by making dequantization cost only two arithmetic instructions per four values and by overlapping loading, dequantization, and tensor-core math without software barriers. The authors report up to 2.9x faster W4A8 kernels than QServe and up to 1.6x over TensorRT-LLM's quantized kernels.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Accuracy of LQQ is asserted but not measured; the serving-system claim depends on it.","rationale":"The reader's weakest-assumption analysis correctly identifies the missing accuracy evaluation as the key unverified premise. The kernel-level claims—LiquidQuant's overflow-safe two-instruction dequantization, the dual-MMA packed layout, and the ImFP pipeline—are supported by detailed argument and plausible benchmark results; I do not find an internal inconsistency in the overflow-safety proof, provided the scale factor is integerized as the IMAD-based implementation requires. However, the end-to-end claim that W4A8 with LQQ is practical for LLM serving depends on the quantized models retaining acceptable quality. The paper explicitly defers the accuracy tables, so the central system-level contribution is conditional on results the reader cannot check. The abstract's 4.94x end-to-end speedup is also not clearly backed by Table 1 or the figures, but that is a presentation/verifiability issue secondary to the accuracy gap. A conditional verdict is therefore appropriate: the kernel contribution may be real, but the system-level claim should not be accepted as fully supported until LQQ accuracy is reported. I recommend no change to the reader's verdict.","tokens_in":20638,"tokens_out":10506,"duration_ms":127067,"concrete_test":"Run LQQ with the paper's default group size (64) on LLaMA2-7B, LLaMA2-70B, LLaMA3-8B, Mistral-7B, and Mixtral-8x7B, and report WikiText2 perplexity and zero-shot accuracy on PIQA, ARC, HellaSwag, and WinoGrande. Compare against FP16 and against QServe's W4A8 quantization (group size 128) on the same benchmarks. The concern is resolved if LQQ is within roughly 0.5 perplexity of QServe and within QServe's published degradation relative to FP16; it lands if LQQ shows substantially larger degradation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central practical claim—that LiquidGEMM makes W4A8 serving viable—rests on LQQ preserving model accuracy. Section 7.1 states only that 'Results show that LQQ preserves accuracy' and defers all perplexity and zero-shot results to a future technical report. This is not a cosmetic omission. LQQ is a new quantization scheme: it shifts INT8 weights by min(Q_i8), rounds the per-group scale to an integer suitable for IMAD, and applies an XOR-based correction. Each of these choices changes quantization error relative to QServe and other W4A8 baselines. If LQQ-dequantized models are meaningfully less accurate than QServe's W4A8 models, then the 2.90x kernel speedup and the LiquidServe throughput numbers in Table 1 do not establish a usable production serving system. The bridge from 'fast kernel' to 'practical W4A8 serving' is currently an unsupported assertion. A secondary gap is that the abstract's 'up to 4.94x end-to-end speedup' is not clearly traceable to any presented experiment, but the missing accuracy evidence is the more load-bearing issue because it invalidates the entire system-level contribution if it fails.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents LiquidGEMM, a W4A8 GEMM kernel for LLM serving on Hopper-class GPUs. It proposes LiquidQuant (LQQ), a two-level quantization scheme that shifts INT8 weights into the UINT8 domain before quantizing to UINT4, and dequantizes with two native instructions (IMAD and XOR) per four elements. A mathematical proof is given that the dequantization computation remains overflow-free within UINT8. The paper also designs an implicit fine-grained pipeline (ImFP) with one load warp group and two compute warp groups to overlap TMA loading, CUDA-core dequantization, and Tensor-Core MMA without explicit synchronization, and a dual-MMA packed SMEM layout that makes LDS.128 efficient for 4-bit weights. Experiments on H800 compare LiquidGEMM with QServe and TensorRT-LLM kernels and report up to 2.90x kernel speedup and up to 4.94x end-to-end system-level speedup, while the accuracy evaluation of LQQ is deferred to a future report.","tokens_in":20954,"tokens_out":7216,"duration_ms":89545,"significance":"If the kernel results are reproducible, the dequantization scheme is a genuine contribution: the overflow-freedom proof in Section 4 is self-contained, the IMAD+XOR formulation directly attacks the identified CUDA-core bottleneck, and the ablation study cleanly separates the effects of LQQ, ExCP, and ImFP. The cost model also provides transparent, falsifiable thresholds on per-element instruction cost. However, the paper's system-level claims are currently incomplete: the accuracy of LQQ is asserted but not measured, and the abstract's 4.94x end-to-end speedup is not traceable to any experiment in Section 7. The kernel may be fast, but the paper has not yet established that it enables a usable W4A8 serving system.","major_comments":[{"comment":"The paper states 'Results show that LQQ preserves accuracy' and then defers all perplexity and zero-shot results to a future technical report. This is load-bearing for the abstract's end-to-end serving claims and for Table 1: if LQQ's quantization error is materially worse than QServe's, the 'up to 4.94x end-to-end speedup' does not establish a practically viable W4A8 serving system. Please include the accuracy tables (WikiText2 perplexity and zero-shot accuracies for the models cited) in this manuscript, or explicitly rescope the claims to kernel-level performance only.","section":"Section 7.1, Experiment Roadmap"},{"comment":"The abstract claims 'up to 4.94x end-to-end system-level speedup', but no experiment in Section 7 supports this number. Table 1 reports speedups of 0.98x to 1.63x relative to the best baseline, and Section 7.2 reports a 1.13-1.98x speedup over LiquidServe/wo. The 4.94x figure appears in the abstract and conclusion only. Please identify the specific experiment and baseline that produce this number, or correct the claim. Without traceability, the headline result is unsupported.","section":"Abstract and Section 9"},{"comment":"The scale factor s_u8 is not defined as an integer, yet the dequantization formula (Eq. 12) relies on an integer multiply for IMAD and the proof relies on s_u8 <= 16. The text first states s_u8 <= floor((119-(-119))/15) = 16, but floor(238/15) = 15; the value 16 comes from rounding to nearest. Please specify the exact integer rounding rule for s_u8 (e.g., s_u8 = round(max(Q_u8)/15) or floor(...)), and reconcile the bound accordingly. This matters both for the overflow-free guarantee and for the quantization error of LQQ.","section":"Section 4, Eq. (7)"}],"minor_comments":[{"comment":"The abstract says 'two arithmetic instructions per four elements', but Section 5.3 says that including the unpacking step, eight elements are dequantized with seven instructions. Please clarify whether the 'two instructions' counts only the IMAD+XOR dequantization after unpacking or the entire per-element cost; the current wording is misleading.","section":"Abstract and Section 5.3"},{"comment":"The notation in Eq. (10) uses floor/ceil-style rounding on q_u/s_u8, but the relationship between Q_u4, Q_u8, and s_u8 in Eq. (7) should be stated more precisely. In particular, state explicitly that quantization uses rounding to nearest integer and that s_u8 is the rounded integer scale.","section":"Section 4, Eq. (10)"},{"comment":"QServe's throughput for Mistral-7B and Yi-34B is much lower than for other systems (e.g., 5,240 and 5,361 tokens/s). Please clarify whether QServe supports these models in its public implementation or whether its W4A8 path is unoptimized for them; otherwise the comparison may overstate LiquidServe's advantage on those rows.","section":"Section 7.2, Table 1"},{"comment":"The motivation and cost model quote H100 hardware metrics, but all experiments are on H800. H800 has different NVLink and some different hardware characteristics; please state whether the Tensor Core and CUDA Core throughputs used in the cost model are the same on H800, or provide H800-specific numbers.","section":"Section 3.1 / Figure 1"},{"comment":"The GEMM latency results are stated as averages over five runs, but no error bars or per-run variance are reported. Please add a sentence on variance or include error bars, especially for the claimed 2.90x speedup at batch size 256.","section":"Section 7.3, Figure 12"}],"recommendation":"major_revision","confidential_remarks":"The core kernel contribution is promising and the overflow proof appears sound. The main blocker is the missing accuracy evaluation and the unsupported 4.94x claim; both are fixable within the manuscript's scope, so I recommend major revision rather than rejection. Please ask the authors to add concrete accuracy numbers or to explicitly limit all serving-level claims to kernel throughput."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is a real engineering contribution. The LQQ dequantization — shift INT8 to UINT8, then IMAD+XOR to get back — is a neat trick that provably avoids overflow, and the proof in Section 4 checks out. The ImFP pipeline, where compute warp groups consume fine-grained tasks without software sync, is also a sensible way to overlap dequantization and MMA. The kernel numbers look plausible: up to 2.9x over QServe's W4A8 kernel at batch 256, and 1.1–1.6x over TensorRT-LLM quantized GEMMs. The ablation study shows LQQ and ImFP each contribute. This is a solid systems paper.\n\nThe soft spot is the one that matters for LLM serving: the paper never shows accuracy. Section 7.1 says 'Results show that LQQ preserves accuracy' and defers all perplexity and zero-shot numbers to a future report. That's not a cosmetic omission. LQQ changes the quantization formula relative to QServe, and if it costs more than a fraction of a perplexity point, the speedups don't translate into a usable serving system. The kernel can still be fast; that claim stands on its own. But the paper frames itself as making W4A8 serving practical, and that bridge is currently unsupported.\n\nThe abstract's 'up to 4.94x end-to-end speedup' is traceable if you compute the ratio of LiquidServe to QServe on Yi-34B from Table 1, but the text doesn't point to that explicitly, and the more honest summary is that the paper shows up to 1.6x over the best baseline. Minor issue.\n\nNo code release is a real limitation, given that the authors already host baselines on GitHub. The complexity of the kernel means independent reproduction would be hard without it.\n\nOverall: the technical core is sound and the engineering is credible. The missing accuracy data is the load-bearing gap. If a referee asks for the accuracy tables and gets them, this becomes a strong paper. If they're not available, the system-level claims should be scaled back to kernel-only performance.\n\nI'd recommend sending to peer review with a request for accuracy data and a clarified abstract. It's a good paper to have in the literature, but as submitted the headlining claim outruns the evidence.","headline":"Solid kernel paper with a genuinely clever dequantization trick; the missing accuracy numbers are the one thing that keeps it from being citable as a serving system.","tokens_in":21447,"tokens_out":3606,"would_cite":false,"duration_ms":36824,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LiquidGEMM reduces W4A8 dequantization to two arithmetic instructions per four elements, unlocking up to 2.90x kernel speedup.","keywords":["W4A8 quantization","GEMM kernel","LLM serving","dequantization","Tensor Cores","implicit pipeline","integer quantization","GPU kernel optimization"],"falsifier":"Run the deferred accuracy evaluation: measure WikiText2 perplexity and the listed zero-shot tasks for LQQ-quantized LLaMA2-7B, LLaMA2-70B, and Mixtral-8x7B against QServe's W4A8 numbers; if perplexity rises significantly or zero-shot accuracy drops, the serving claim collapses. Separately, disassemble the shipped kernel to confirm dequantization compiles to one IMAD and one XOR per four elements and that no software barriers appear in the main loop.","tokens_in":1743,"feed_emoji":"⚙️","tokens_out":1647,"duration_ms":66944,"temperature":0.7,"pith_summary":"The paper argues that W4A8 quantization—4-bit weights, 8-bit activations—can finally deliver its theoretical speed advantage in LLM serving, but only if the dequantization step stops competing with the GPU's fast Tensor Cores. It claims that prior W4A8 kernels spend so many instructions recovering 8-bit values from packed 4-bit weights on the slower CUDA cores that the quantization advantage disappears, sometimes making W4A8 slower than W8A8. LiquidGEMM's two contributions are LiquidQuant, a rotation-based quantization scheme that makes dequantization overflow-safe in exactly two arithmetic instructions (IMAD and XOR) per four elements, and an implicit fine-grained pipeline that overlaps weight loading, dequantization, and matrix multiply without explicit synchronization. If correct, dequantization is no longer the bottleneck, and W4A8 becomes practical for production serving, with reported kernel speedups of up to 2.90x over the leading W4A8 kernel and 1.12–1.63x over TensorRT-LLM's quantized kernels. The paper also reports up to 4.94x end-to-end system-level speedup, while deferring the model-accuracy evidence that would make the whole system viable.","feed_headline":"Two arithmetic instructions unlock 2.9x faster W4A8 GEMM","feed_subtitle":"Overflow-safe dequantization plus an implicit pipeline keeps Tensor Cores fed, up to 4.94x end-to-end speedup.","key_machinery":"LiquidQuant's sweet dequantization identity (q_u4 * s_u8 + a) XOR 0x80, where a = 2^7 + min(Q_i8) is precomputed offline, carries the arithmetic: it turns an overflow-prone signed add into an unsigned multiply-add plus a most-significant-bit flip, provably staying inside UINT8 and executable by native IMAD and XOR. The implicit fine-grained pipeline (ImFP) carries the execution: a single-producer, multiple-consumer model where one load warp group feeds shared memory and several compute warp groups each dequantize and run MMA, using hardware task arbitration instead of software barriers to overlap loading, dequantization, and tensor-core work.","core_discovery":"The paper's central claim is that the dequantization step, not the tensor-core MMA, is what makes W4A8 GEMM underperform, and that both halves of the problem—the arithmetic and the scheduling—can be redesigned to remove it. LiquidQuant re-quantizes weights so that decoding UINT4 back to INT8 reduces to (q_u4 * s_u8 + a) XOR 0x80, a multiply-add plus a bit flip that stays in the UINT8 range by construction, costing two hardware instructions per four elements instead of the instruction-heavy overflow handling used by prior kernels. The implicit fine-grained pipeline assigns one warp group to load weights and several compute warp groups to dequantize and immediately run MMA on them, letting har","pith_inferences":["If the deferred accuracy report confirms that LQQ matches prior W4A8 accuracy, the kernel speedups translate directly into production gains; if it does not, the kernel is fast but the serving system is not viable as claimed.","The same bit-flip dequantization trick should generalize to other asymmetric low-bit formats, such as W3A8 or per-group UINT4 on other GPUs, because it relies only on two's complement arithmetic and a bounded protective range.","The implicit pipeline style—hardware-arbitrated task consumption instead of software barriers—could be applied to other mixed-precision kernels where a cheap operand transformation sits between load and MMA.","The cost model implies that as Tensor Core throughput grows faster than memory bandwidth, keeping per-element dequantization cost small becomes more important; LiquidQuant's constant two-instruction cost lowers the batch size at which W4A8 becomes compute-bound."],"forward_implications":["W4A8 GEMM no longer needs a dedicated dequantization warp group; the same warp can dequantize and directly feed Tensor Cores.","The kernel scales better at large batch sizes than prior W4A8 kernels, because dequantization is overlapped rather than serialized.","In memory-bound small-batch scenarios, W4A8's lower memory traffic can beat W8A8 and FP8; in compute-bound large-batch scenarios it can match or beat them.","System-level serving throughput improves up to 4.94x over QServe and up to 1.63x over TensorRT-LLM, with the largest gains on large models like LLaMA2-70B.","LiquidQuant is orthogonal to activation-smoothing and accuracy-oriented quantization methods, so it can be combined with them rather than competing."],"supporting_citations":[{"why":"Supplies the W4A8 kernel and QoQ dequantization method that LiquidGEMM must beat, plus the protective quantization range and unpacking approach reused here.","marker":"[15]"},{"why":"Provides the TensorRT-LLM baseline kernels (FP16, W4A16, W8A8, FP8) used for kernel and system-level comparison.","marker":"[20]"},{"why":"Supplies the SmoothQuant activation quantization and smooth-scale scheme used in the end-to-end serving system.","marker":"[29]"},{"why":"Prior W4A8 kernel and roofline analysis used as context for why existing W4A8 implementations underperform.","marker":"[34]"},{"why":"Prior low-bit GEMM kernel considered and benchmarked as a baseline, then omitted because INT4 tensor cores are unsupported on H800.","marker":"[35]"},{"why":"Provides PagedAttention for KV cache management in the end-to-end serving system evaluation.","marker":"[12]"}],"fun_headline_variants":["Two-instruction dequantization makes W4A8 GEMM 2.9x faster","Implicit pipeline keeps Tensor Cores fed, speeding W4A8 2.9x","LiquidGEMM: 2.9x faster W4A8 kernels without overflow checks","W4A8 GEMM gets 4.94x system speedup via two-instruction dequant","The bottleneck is dequantization, not Tensor Cores—LiquidGEMM fixes it"],"cache_read_input_tokens":23168,"weakest_assumption_plain":"The argument's load-bearing premise is that LiquidQuant's W4A8 models keep accuracy close to previous W4A8 methods; the paper asserts this but withholds the accuracy tables (Section 7.1), so the kernel speedups only matter if that assertion holds.","fun_headline_variants_meta":{"raw":{"variants":["Two-instruction dequantization makes W4A8 GEMM 2.9x faster","Implicit pipeline keeps Tensor Cores fed, speeding W4A8 2.9x","LiquidGEMM: 2.9x faster W4A8 kernels without overflow checks","W4A8 GEMM gets 4.94x system speedup via two-instruction dequant","The bottleneck is dequantization, not Tensor Cores—LiquidGEMM fixes it"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000242,"raw_usage":{"total_tokens":1388,"prompt_tokens":800,"completion_tokens":588,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":544,"completion_tokens_details":{"reasoning_tokens":480}},"tokens_in":544,"tokens_out":588,"duration_ms":5785,"temperature":1.0,"reasoning_tokens":480,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T12:44:32.607766+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the deferred accuracy evaluation: measure WikiText2 perplexity and the listed zero-shot tasks for LQQ-quantized LLaMA2-7B, LLaMA2-70B, and Mixtral-8x7B against QServe's W4A8 numbers; if perplexity rises significantly or zero-shot accuracy drops, the serving claim collapses. Separately, disassemble the shipped kernel to confirm dequantization compiles to one IMAD and one XOR per four elements and that no software barriers appear in the main loop.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the TensorRT-LLM baseline kernels (FP16, W4A16, W8A8, FP8) used for kernel and system-level comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior low-bit GEMM kernel considered and benchmarked as a baseline, then omitted because INT4 tensor cores are unsupported on H800."}],"review_version":1}