{"id":"d8e7222f-cf81-4a61-b3d7-cd0b18682ee5","arxiv_id":"2504.12984","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A tile-level GPU language with an algebraic layout system generates kernels for arbitrary 1-8 bit quantized types, outperforming Triton, Ladder, QuantLLM, and Marlin on supported workloads.","lead":"Tilus is a new programming language for writing GPU kernels that natively handle weights quantized to any width from 1 to 8 bits. It reports speedups of up to 2.6x over existing compilers and hand-tuned kernels for low-precision LLM serving.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No-cost View (Fig. 2c, §7.2) is the load-bearing efficiency claim, but 'same bits per thread' does not guarantee a zero-instruction reinterpretation unless physical bit-packing is specified and preserved by codegen; this is asserted, not shown.","rationale":"The reader's weakest assumption is precisely the concern I find most load-bearing. The paper's headline contribution is that arbitrary sub-byte types are supported without the shared-memory layout conversions that dominate Triton's low-precision kernels; Figure 1 and Figure 2 present View as the enabling operation, and Section 7.2 extends it to all low-precision layouts. If View is not actually free for some widths, the claimed 1.75x/2.61x advantages shrink to whatever remains after bit-packing overhead, and the 'arbitrary widths' claim reduces to 'arbitrary widths with extra instructions.' I checked the other candidates: the abstract's 'limited to powers of two' overstates the related work (the paper itself cites arbitrary-precision efforts [3, 18, 35, 45, 64]), but that is a novelty/positioning issue, not a direct threat to the measured numbers. Missing error bars affect the 1.03x Marlin comparison but not the structural argument. The no-cost View concern is concrete, testable from the shipped artifact, and consistent with the paper's own language: the layout system defines logical-to-thread mapping but never defines the bit-level register packing that makes reinterpretation a bijection. The paper is not internally inconsistent, and an implementation maintaining packed bitstring registers would satisfy the claim; the artifact makes this check feasible. I therefore keep the reader's CONDITIONAL verdict rather than escalating.","tokens_in":28908,"tokens_out":10521,"duration_ms":109314,"concrete_test":"Use the released artifact to compile the FP16×INT6 matmul of Fig. 2 and one non-power-of-two kernel from Fig. 11 (int5 or int7); emit the generated CUDA/Hidet IR and disassemble the SASS around the View instruction (line 11 in Fig. 2). Count target instructions between the last memory load and the first PRMT/LOP3 of the Cast. If View lowers to zero instructions and the payload occupies one contiguous register bitstring, the concern does not land. If any shift/mask/or is emitted, the 'no cost' guarantee fails for that layout and the Triton comparison must be re-attributed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central speedup story (Fig. 1 and §3.2) is that Tilus eliminates the shared-memory layout conversion by reinterpreting a register tensor from u8[3]/thread to i6[4]/thread, or from any low-precision layout to a hardware-friendly u8 load layout, at no cost. The only compatibility conditions given (Fig. 2c) are same thread count and same total bits per thread. Those conditions are not sufficient if the physical register representation is not a single contiguous, ordered bitstring. The layout algebra of §4/§5 maps a logical index to (thread t, local index i), but it does not state how the i local elements are placed in registers, what bit offset each element starts at, or how padding/alignment is handled. For standard types a local element normally occupies one 32-bit register; if the same representation is used for sub-byte types, converting 3 u8 to 4 i6 requires moving bits between registers (shifts/PRMT/LOP3), making View costly. The paper's §7.2 general formula chooses a u8 layout based only on bytes per thread and never proves that an arbitrary target layout is bit-compatible with it. Non-byte-aligned cases (e.g., u1 with 4 bits/thread) cannot be re-expressed as u8 without changing per-thread bit counts or grouping threads, neither of which is specified. Thus the mechanism that supports 'arbitrary 1-8 bit' performance is asserted and demonstrated on selected examples, not established for all layouts. This is a verifiable implementation question, not an obvious contradiction; the open-source artifact can settle it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Tilus, a tile-level GPGPU programming language with an algebraic layout system for register tensors, a thread-block-level virtual machine instruction set, and native support for low-precision data types with arbitrary bit widths from 1 to 8. The central performance claim is that a View instruction can reinterpret register tensors between different data types and layouts at zero cost when the number of threads and the total number of bits per thread match, thereby eliminating the shared-memory layout conversions that dominate Triton-style low-precision kernels. The paper reports speedups over Triton, Ladder, QuantLLM, and Marlin, includes end-to-end LLM serving results across three models and three GPU architectures, and provides open-source artifacts with a Docker workflow.","tokens_in":29205,"tokens_out":7319,"duration_ms":71882,"significance":"If the zero-cost View claim is valid, Tilus is a significant contribution: it demonstrates that a single parameterized program template can generate efficient kernels for the full spectrum of 1-8 bit quantization formats, a capability that existing compilers and hand-written kernels do not provide at this level of generality or performance. The algebraic layout system is a clean formalization with no fitted constants (the only free parameters are tile sizes), and the artifacts are open-sourced with a reproducible benchmark protocol, including median-of-50 operator runs, median-of-10 model runs, L2 cache clearing, and auto-tuned baselines. The reported speedup over Marlin, the strongest hand-tuned baseline, is modest (1.03x), but the breadth of supported data types is the main value proposition. The main risk to significance is that the zero-cost View instruction is the mechanism behind the efficiency gains, and its semantics are not sufficiently specified to establish that the cost is truly zero.","major_comments":[{"comment":"The zero-cost View claim is load-bearing for the reported speedups but is not established. The compatibility rule in Fig. 2(c) — same number of threads and same total bits per thread — is insufficient unless the physical bit-level register layout is specified. The layout algebra in Section 4 maps a logical index to a (thread, local-index) pair, but it never defines the bit offset, ordering, or padding of sub-byte elements within a thread's registers. Consequently, reinterpreting 3×u8 per thread as 4×i6 per thread may require bit extraction and reassembly instructions (shifts, LOP3, PRMT) unless the bit patterns happen to align. Table 1 defines View as \"at no cost\" by fiat, but no proof or codegen-level analysis is provided. Please specify the register representation and either prove bit-compatibility for the layouts used in Fig. 2 and Section 7.2, or present the emitted PTX/SASS to demonstrate that View compiles to zero instructions for these cases.","section":"§3.2, Fig. 2(c), Table 1"},{"comment":"The general method for efficient low-precision loading is only defined for layouts where each thread holds an integer number of bytes. The formula n1 = gcd(n, 16), n2 = n / gcd(n1, 16) requires an integer n (bytes per thread). The paper claims support for arbitrary 1-8 bit widths and benchmarks u1 and u2 in Figure 10, but for per-thread bit counts that are not multiples of 8 (e.g., 4 bits per thread for u1 in a 32-thread warp), this u8-reinterpretation recipe does not apply. The text states that the method \"generalizes to loading any low-precision tensor with arbitrary layout,\" but it does not explain how non-byte-aligned per-thread sizes are handled. Please extend the construction to non-integer bytes per thread, or clarify the alternative mechanism used for the u1/u2 benchmarks, and show the corresponding View layouts for those cases.","section":"§7.2"},{"comment":"The headline aggregate speedups (1.75x, 2.61x, 1.29x, 1.03x) are not defined: the paper does not specify whether these are means, medians, or maxima over the workloads in Figure 10, and no variance information is reported anywhere. Since 1.03x against Marlin is close to plausible run-to-run noise, the comparison would be much more convincing with per-configuration distributions or confidence intervals. In addition, the end-to-end experiments use dummy weights (as stated in A.3.4), so the paper contains no accuracy validation of the quantized models; this should be stated prominently in the main text, since the motivation in Section 2.1 concerns the accuracy-efficiency trade-off of 5-7 bit quantization.","section":"Abstract, §9.1-§9.2, §A.3.4"}],"minor_comments":[{"comment":"The Kronecker product definition uses the symbol ⊙ for element-wise product without defining it explicitly for index vectors; a reader familiar with matrix operations may confuse it with the Hadamard product. Please add one sentence clarifying that ⊙ operates component-wise on the logical index.","section":"Section 4.2"},{"comment":"References [56] and [57] are both \"Attention is All you Need\" by Vaswani et al.; the duplicate should be removed or replaced with the intended citing work.","section":"References"},{"comment":"The speedup labels on the bars are difficult to read, especially for small values; consider reporting the exact numbers in a table or using a different labeling scheme.","section":"Figure 10"},{"comment":"The description of the unified layout representation would benefit from a worked example showing how mode_shape, spatial_modes, and local_modes are derived from a given Kronecker-product expression; the current example in Figure 6 is not explicitly connected to the primitives of Section 4.","section":"Section 5"},{"comment":"The statement that existing approaches for generating low-precision kernels are limited to power-of-two bit widths is too strong, given that the related work section itself cites APNN-TC, Any-Precision LLM, and ABQ-LLM, which support arbitrary bit widths; please add a qualifier such as \"efficiently\" or \"in a compiler setting.\"","section":"Section 1"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the semantic gap in the View instruction: the paper's formal layout system abstracts away the physical bit placement in registers, yet the zero-cost reinterpretation claim depends on exactly that placement. This is fixable by adding a bit-level register layout definition and either a proof or SASS-level evidence. The paper is otherwise well-executed, with a strong artifact and reproducible methodology. I also note that the manuscript appears to be a camera-ready revision of an ASPLOS submission (per the acknowledgements); the editor may want to confirm the submission venue and novelty disclosure relative to that version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about Tilus. First, it is a genuinely new compiler-based approach to sub-byte quantization kernels: a tile-level DSL with an algebraic layout system built from Kronecker products, a unified layout representation, and compiler support for arbitrary 1–8 bit signed, unsigned, and floating-point types. Second, the paper ships: code, Dockerized artifacts, and benchmarks against strong baselines (Triton, Ladder, QuantLLM, Marlin, cuBLAS). If the numbers hold up on a careful artifact pass, this closes a real gap for 5/6/7-bit LLM serving.\n\nWhat is actually new is the layout algebra. The Kronecker-product composition and the unified mode-based representation give a clean way to reason about thread-level data placement, and the paper shows it can express tensor-core layouts precisely. The thread-block-level VM with explicit memory hierarchy is a reasonable design choice, and the automatic vectorization and instruction selection are sensible. The benchmark suite is broad: multiple GPUs, multiple bit widths, multiple data type kinds, and end-to-end LLM serving. The open-source artifacts are a real asset.\n\nThe soft spot I would probe first is the no-cost View. The paper's compatibility conditions are same thread count and same total bits per thread. That is not sufficient in general: if the physical register file does not lay out sub-byte elements as one contiguous bitstring, the reinterpretation requires PRMT/LOP3/shifts, which are not free. Section 7.2 gives a formula for choosing a u8 layout but never proves the target layout is bit-compatible with it, and the case where bytes per thread is not an integer (e.g., u1 with 4 bits/thread) is not handled in the general statement. This is a verifiable implementation detail, and the artifacts can settle it; I would not call it fatal, but the claim as written is stronger than what is shown. Minor issues: no error bars or variance on the latency numbers, no accuracy validation for the quantized models (not strictly required, but relevant for the 5–7 bit motivation), and the abstract overstates that prior approaches are limited to power-of-two widths when the related work section itself cites APNN-TC, eXmY, ABQ-LLM, and Any-Precision LLM. The latter is an easy fix.\n\nWho is this for? ML systems researchers working on quantization-aware kernel generation, LLM serving stacks, and GPU compiler folks. It deserves a serious referee; I would send it out and ask for an artifact audit plus a clarification of the View guarantee before accepting. A revision that either proves the bit-compatibility condition for the layouts it claims, or scopes the claim to the layouts where it is actually free, would make this a strong paper.","headline":"Tilus is a solid, well-engineered compiler paper for sub-byte quantization kernels with real artifacts; the main soft spot is an unproven no-cost register reinterpretation guarantee, not a showstopper.","tokens_in":29800,"tokens_out":2373,"would_cite":true,"duration_ms":24880,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Tilus is a tile-level GPGPU language that turns any 1-to-8-bit low-precision tensor type into kernels that beat existing compilers and hand-tuned code.","keywords":["GPU programming language","low-precision quantization","arbitrary bit width","algebraic layout system","register tensors","tensor cores","LLM serving","kernel compilation"],"falsifier":"Compile a Tilus program whose body is only LoadGlobal(u8), then View as an i6 layout, then StoreGlobal, and run it on a fixed GPU with a fixed tile; compile the same program with the View replaced by an explicit copy through shared memory. If the View version is not measurably faster or equal, and its generated machine code contains register moves, shuffles, or extra memory instructions for any legal layout, the zero-cost reinterpretation claim fails for that layout.","tokens_in":28679,"feed_emoji":"⚙️","tokens_out":7434,"duration_ms":74382,"temperature":0.7,"pith_summary":"This paper introduces Tilus, a tile-level GPGPU language, and argues that arbitrary low-precision data types—integers and floats from 1 to 8 bits—can be made both easy to program and fast on GPUs. Its central claim is that a language exposing thread-block-level instructions, hierarchical memory, and an algebraic layout system can eliminate the shared-memory layout conversions and missing software pipelining that make odd-width quantization (3, 5, 6, 7 bits) slow today. If true, the practical accuracy-efficiency trade-off for quantized LLM serving widens: 4-bit quantization is no longer the only fast option, and hand-written kernels for each new width become unnecessary. The paper reports speedups of 1.75x over a leading tile-oriented compiler, 2.61x over a schedule-oriented low-precision compiler, and 1.29x and 1.03x over hand-tuned 5/6-bit and 4-bit kernels, all from one parameterized kernel template.","feed_headline":"1-8 bit quantization runs up to 2.61x faster with Tilus","feed_subtitle":"A register-level View replaces shared-memory layout conversion in low-precision LLM serving.","key_machinery":"The central object is the register tensor layout, defined as a function f(t,i) that maps thread index t and local element index i to the logical index of the tensor element. Tilus builds this function from primitive local(n1,n2) and spatial(n1,n2) layouts combined by a Kronecker product, giving a unified representation in terms of mode_shape, spatial_modes, and local_modes. The load-bearing operation is the View instruction: it reinterprets a register tensor's data type and layout whenever the number of threads and the number of bits per thread match, making sub-byte reinterpretations a zero-cost register-level operation. This is what lets Tilus load arbitrary low-precision tiles as hardware-friendly uint8 and defer type conversion to vectorized casts.","core_discovery":"Tilus's discovery is that the layout of a register tensor—the mapping from a thread index and a local element index to the logical tensor index—can be composed algebraically out of two primitives, local and spatial, using a Kronecker product. Because these layouts are closed under the product, any layout can be written in a unified form, and two register tensors with the same number of threads and the same number of bits per thread can be reinterpreted from one data type and layout to another with a View instruction that costs nothing. This turns low-precision weight loading into a two-step pipeline—load the packed bytes as uint8, then View them as int6, int5, or float6—replacing the shared-memory layout conversion that tile-oriented compilers need and the type-level packing that limits schedule-oriented compilers to power-of-two widths. Casting from the low-precision type to float16 is then done entirely in registers with vectorized bit-manipulation instructions. On this basis Tilus reports up to 2.61x speedup over the schedule-oriented baseline and beats hand-tuned kernels for the widths those kernels support.","pith_inferences":["Beyond the paper's GPU matmul demonstrations, the bit-per-thread reinterpretation argument is width-invariant, so the same View strategy should extend to 9-to-15-bit types packable into 16-bit lanes and to non-GPU vector hardware.","Because layouts are closed under the Kronecker product and View is a bijection on the (thread, bit) space, the layout algebra could double as a verification target: one could mechanically prove that a View produces the logical tensor content requested, catching register-allocation bugs the compiler misses.","The smallest reported margin is 1.03x against a hand-tuned 4-bit kernel, which suggests the remaining gap to expert kernels is thin; sustaining it on new architectures will depend on the View remaining free in generated machine code.","The load-reinterpret path is not matmul-specific, so a natural test is whether Tilus extends to activation quantization with mixed per-tensor widths or to attention kernels, which the paper does not evaluate."],"forward_implications":["A single Tilus program template covers uint1-uint8, int2-int8, and float3-float8, so new quantization widths need no new hand-written kernels.","Odd widths like 3, 5, 6, and 7 bits become viable in LLM serving, allowing accuracy-preserving quantization where 4-bit is too aggressive and 8-bit wasteful.","Weight loading for low-precision matmul can be software-pipelined via asynchronous copies and register-level reinterpretation, removing the shared-memory layout-conversion bottleneck.","End-to-end decode latency at small batch sizes improves across 9B, 32B, and 70B models on A100, L40S, and H100 GPUs.","The same virtual-machine instruction set allows target-specific instruction selection, so new GPU features can be added without rewriting kernels."],"supporting_citations":[{"why":"The tile-oriented compiler baseline: its lack of native low-precision types and hidden layouts is the gap Tilus is designed to close, and its measured latency supplies the 1.75x comparison.","marker":"[53]"},{"why":"The schedule-oriented low-precision baseline whose type-level packing only handles power-of-two widths; supplies the 2.61x comparison and the missing-pipelining analysis.","marker":"[58]"},{"why":"Provides the hand-tuned 5/6-bit float kernels used for the 1.29x comparison.","marker":"[60]"},{"why":"Provides the hand-optimized 4-bit kernel baseline for the 1.03x comparison.","marker":"[21]"},{"why":"Defines the tensor-core instruction layouts and shared-memory load instructions that Tilus's layout algebra expresses and targets.","marker":"[41]"},{"why":"Backend IR used to emit low-level GPU code; central to the compiler implementation.","marker":"[14]"},{"why":"Defines load-compute pipelining, the optimization whose absence in the schedule-oriented baseline and presence in Tilus accounts for part of the speedup.","marker":"[26]"},{"why":"Provides the memory hierarchy, instruction set, and compilation details (async copies, vector loads, nvcc) that code generation and evaluation rely on.","marker":"[40]"}],"fun_headline_variants":["Tilus: arbitrary bit widths with zero-cost Views","Tilus speeds up low-precision LLM serving by 2.61x","Zero-cost layout view powers up to 2.61x faster kernels","Arbitrary 1-8 bit types at 2.61x speedup with Tilus","Kronecker layouts enable free type views in low-precision GPGPU"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that two register tensors with the same number of threads and the same number of bits per thread can always be reinterpreted from one data type and layout into another at zero runtime cost; if register allocation or bit ordering makes that reinterpretation cost real for some layouts, the performance claims for those widths would shrink even though the language would still work.","fun_headline_variants_meta":{"raw":{"variants":["Tilus: arbitrary bit widths with zero-cost Views","Tilus speeds up low-precision LLM serving by 2.61x","Zero-cost layout view powers up to 2.61x faster kernels","Arbitrary 1-8 bit types at 2.61x speedup with Tilus","Kronecker layouts enable free type views in low-precision GPGPU"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000839,"raw_usage":{"total_tokens":3718,"prompt_tokens":1070,"completion_tokens":2648,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":686,"completion_tokens_details":{"reasoning_tokens":2548}},"tokens_in":686,"tokens_out":2648,"duration_ms":15662,"temperature":1.0,"reasoning_tokens":2548,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:18:18.563331+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compile a Tilus program whose body is only LoadGlobal(u8), then View as an i6 layout, then StoreGlobal, and run it on a fixed GPU with a fixed tile; compile the same program with the View replaced by an explicit copy through shared memory. If the View version is not measurably faster or equal, and its generated machine code contains register moves, shuffles, or extra memory instructions for any legal layout, the zero-cost reinterpretation claim fails for that layout.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The schedule-oriented low-precision baseline whose type-level packing only handles power-of-two widths; supplies the 2.61x comparison and the missing-pipelining analysis."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the hand-tuned 5/6-bit float kernels used for the 1.29x comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the tensor-core instruction layouts and shared-memory load instructions that Tilus's layout algebra expresses and targets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines load-compute pipelining, the optimization whose absence in the schedule-oriented baseline and presence in Tilus accounts for part of the speedup."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the memory hierarchy, instruction set, and compilation details (async copies, vector loads, nvcc) that code generation and evaluation rely on."}],"review_version":1}