{"id":"56629b72-9ed5-46b6-a91d-753c1007e3ba","arxiv_id":"2607.22432","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A tile-centric analytical model predicts GPU kernel latency and cache behavior purely from microbenchmark-calibrated hardware rates, reaching about 12% GEMM error and 13% end-to-end LLM serving error across five GPU lines.","lead":"TileSight is an analytical tool that predicts GPU kernel runtime—from one chip to a cluster—without training on measured kernel data, using the tile as its core unit. It could let AI-kernel engineers tune and diagnose performance faster than current roofline, machine-learning, or post-hoc profiler approaches.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Uniform-SM reuse-distance assumption (§3.5.3, §5.3) creates systematic L2 hit-rate optimism for deep-K GEMMs; the paper never quantifies its downstream effect on the headline 12.35% GEMM latency MAPE.","rationale":"The reader and I identify the same load-bearing concern: the lockstep-SM assumption in the reuse-distance cache model (§3.5.3) leads to systematic L2 hit-rate optimism for deep-K GEMMs, and the paper does not quantify the downstream effect on the headline latency claim. This is a genuine soft spot because the cache-traffic split is not a peripheral diagnostic—it feeds Eq. (1)'s resource vector and hence every latency prediction. The paper is commendably transparent about the discrepancy in §5.3 and acknowledges it again in §7, which is evidence of good-faith reporting, not a reason to dismiss it. However, transparency is not the same as resolution: the claimed 'within roughly one percentage point' L2 error is an aggregate over 4,680 cases and is contradicted by a 39pp per-config error on a representative deep-K shape. The same bias plausibly affects the 703-GEMM latency MAPE, since large-K GEMMs are common in LLM prefill and in the TileLang cost-model workloads listed in Figure 12. Without a stratification of the 703 shapes by K, or a per-bucket MAPE, the 12.35% pooled number could hide a concentrated weakness. The concrete test I propose would settle this—either by showing that deep-K shapes are rare enough that the pooled number is robust, or by demonstrating that the lockstep assumption introduces a larger error in exactly the regime the paper claims to support. This does not change the reader's verdict: CONDITIONAL remains appropriate. The absence of released code/data is a secondary concern that amplifies the need for the test, since the authors' own measurements are the only way to check it.","tokens_in":24188,"tokens_out":4637,"duration_ms":59435,"concrete_test":"Re-run TileSight on the 703 GEMM shapes, stratifying by K dimension (e.g., K ≥ 8192 vs. K < 8192, or working set exceeding the calibrated L2 capacity). Report the count, per-bucket latency MAPE, and a scatter of predicted vs. measured L2 hit rate for each deep-K shape. If the deep-K bucket shows substantially higher MAPE (e.g., >20%) while the shallow-K bucket is near or below the pooled 12.35%, the headline must be qualified. Additionally, implement a desynchronization correction in the reuse-distance tracker of §3.5.3—e.g., assign each SM an independent K-loop phase offset drawn from a calibrated skew distribution—and re-evaluate the H200 M=N=8192, K=28672 case. If the predicted hit rate moves from 82% toward 43% and the latency error shrinks, the bias is fixable and its magnitude should be quantified; if it does not, the lockstep assumption is not the sole cause and the discrepancy ne","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that a training-free analytical tile model predicts GEMM latency and cache behavior across GPUs from microbenchmark-calibrated rates—rests on the reuse-distance cache model of §3.5. The model assumes tiles advance at a uniform rate across SMs. When SMs desynchronize, as they do in large high-occupancy kernels, the concurrent tile footprint spreads across many K-slices, increasing the effective reuse distance and lowering the L2 hit rate below the lockstep prediction. The paper's own §5.3 reports a deep-K GEMM (M=N=8192, K=28672 on H200) where TileSight predicts 82% L2 hit rate but NCU measures 43%—a 39pp error, not the 'within roughly one percentage point' that the abstract claims. Because the L1.5/L2/DDR traffic fractions from this cache model populate the per-tile resource vector (Eq. 1) and flow into the pipeline envelope (Eqs. 2–5) and wave aggregation, any systematic L2 hit-rate optimism propagates directly into latency predictions when the kernel is memory-side bound. The paper calls this 'mildly optimistic' and 'rare,' but it provides no quantification: it does not report how many of the 703 GEMM shapes fall into the deep-K regime, nor what the latency MAPE is when restricted to those shapes. The workloads in Figure 12—e.g., 8192×28672×8192 and 2048×12288×49152—are exactly this regime, so the central claim is narrower than stated for LLM-relevant GEMMs. The aggregate 12.35% MAPE could mask a concentrated bias.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"TileSight proposes a tile-centric analytical GPU performance model that spans three levels: intra-tile resource-vector modeling, inter-tile scheduling and reuse-distance cache modeling, and cross-device alpha-beta communication modeling. The paper claims that, using only one-time per-architecture microbenchmark calibration, TileSight predicts single-GPU GEMM latency with 12.35% pooled MAPE across 703 BF16/FP16 tensor-core shapes on A100, B200, B6000, and H200, outperforming Roofline, NeuSight, PipeWeave, and GenZ. It further reports L2 hit-rate predictions within about one percentage point MAE on 4,680 persistent-kernel cases, 16.18% wMAPE on 304 distributed fused kernels, 13.52% wMAPE on 166 vLLM decode configurations, and competitive results as a schedule-pruning cost model.","tokens_in":24573,"tokens_out":6523,"duration_ms":79598,"significance":"If the central claims hold, this is a substantial contribution: it provides a white-box, schedule-sensitive, architecture-transferable performance model that matches the tile abstraction used by modern GPU programming frameworks, and it can serve as both a diagnostic tool and an analytical cost model for autotuning. The evaluation is unusually broad in hardware coverage (Ampere, Hopper, Blackwell, and an AMD GPU), workload breadth (GEMM, attention, collectives, fused distributed kernels, end-to-end serving), and comparison to learned and analytical baselines. The methodology is transparent about its calibrated hardware parameters, and the paper is honest about its limitations. The main risk is that the load-bearing cache-reuse model has a known systematic bias in the deep-K regime, and the paper does not quantify this bias's downstream effect on the headline latency MAPE.","major_comments":[{"comment":"The uniform-SM progress assumption is load-bearing. §5.3 reports a GEMM with M=N=8192, K=28672 on H200 where TileSight predicts 82% L2 hit rate vs. 43% measured. Because the L1.5/L2/DDR traffic split computed in §3.5 populates the resource vector (Eq. 1) and flows into the pipeline envelope (Eqs. 2–5), this 39 pp error can bias latency predictions whenever the kernel is memory-side bound. The paper calls the regime 'rare' and 'mildly optimistic' but gives no distribution: how many of the 703 GEMM shapes fall into the deep-K/high-occupancy regime, and what is the latency MAPE restricted to those shapes? The workloads in Figure 12 include 8192×28672×8192 and 2048×12288×49152, which are exactly this regime. Please report the conditional error, and either correct the model or qualify the abstract's 'within roughly one percentage point' and the pooled 12.35% latency claim.","section":"§5.3, §7, Eq. (1)"},{"comment":"The central empirical claims rest on 703 GEMM shapes, 4,680 cache cases, 304 distributed cases, and 166 vLLM configurations, but no code, data, or scripts are provided; the paper only promises open-sourcing 'upon publication.' For a systems paper whose main evidence is a numerical evaluation, this is insufficient for verification. Please release the implementation and the exact workload/config lists, and report run-to-run variance or confidence intervals for the reported MAPEs, since GPU latency measurements are noisy. At minimum, a detailed hardware-parameter table (e.g., effective L2 capacity B_T, per-pipeline calibrated rates) should be included as supplementary material.","section":"§5 (all)"},{"comment":"The pooled 12.35% MAPE aggregates four GPU families with very different per-architecture errors (5.2% on B6000, 18.7% on A100). The paper does not report how the 703 shapes are distributed across architectures, nor whether the pooled number weights each shape equally or each architecture equally. This makes the transferability claim hard to assess. Also, the GEMM evaluation filters out stream-K and SIMT fallback paths; the number of filtered shapes and their behavior should be reported, since these paths are common in production GEMM workloads and are not covered by the headline claim.","section":"§5.2, Fig. 5"}],"minor_comments":[{"comment":"The abstract says 'L2 cache-hit-rate predictions land within roughly one percentage point of the measured rate on every GPU.' This is true as MAE over the 4,680-case sweep, but the text should say 'mean absolute error' and give a quantile or outlier count, because the deep-K example in §5.3 is a 39 pp error on a single configuration.","section":"Abstract and §5.3"},{"comment":"The baseline comparison is transparent but not apples-to-apples: PipeWeave falls back to H800 models for H200/B200, and NeuSight is retrained on PipeWeave's FP16 data. The text should state that the distributed and end-to-end baseline comparisons partly reflect dataset availability rather than only model quality, and consider adding an analytical/simulation baseline for the cache model.","section":"§5.1"},{"comment":"The symbol A is used for cache associativity in Eqs. (7)–(9), which is confusing given the matrix A in GEMM discussions. Rename to 'assoc' or a similar symbol.","section":"§3.5.2"},{"comment":"Reference [12] appears incomplete: 'CUTLASS acce.' is not a valid author string. Reference [15] is a news brief; for a claim about CUDA Tile, consider citing NVIDIA's official documentation instead.","section":"References"},{"comment":"The 'No kernel profiling/training' mark is correct but should be clarified: the model uses per-architecture microbenchmarks for calibration. Without this clarification, 'first-principles' may be read as parameter-free rather than training-free.","section":"§1, Table 1"}],"recommendation":"major_revision","confidential_remarks":"I see this as a strong paper with a clear central idea and unusually broad validation. My main technical concern is the unquantified effect of the uniform-SM assumption on the central latency claim; this is fixable with conditional error reporting or a simple desynchronization heuristic. I am not concerned about circularity in the sense of training on target kernels: the calibration is per-hardware and the cache study is out-of-sample on shapes. I would make code/data release a condition of acceptance, and I would request confidence intervals on the headline MAPE."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a solid systems paper and worth referee time. The core idea — using the tile as the analysis primitive across intra-GPU pipeline modeling, cache hierarchy, and cross-device communication — is genuinely new and well executed. The recursive prologue-steady-epilogue envelope and the tile-granular reuse-distance cache model are the right abstractions, and the validation is impressively broad: 703 GEMM shapes across five architectures, 4,680 cache cases, 304 distributed kernels, and 166 vLLM configurations, with a 12.35% pooled GEMM MAPE and 13.52% end-to-end wMAPE. The model is trained on nothing; calibration comes from microbenchmarks. That is a real contribution, especially the 99.66% of best performance when used to prune TileLang schedules.\n\nWhat is also good is the honesty. The limitations section and §5.3 explicitly flag the uniform-SM assumption and the systematic L2 hit-rate optimism for deep-K GEMMs. The paper reports an 82% predicted vs. 43% measured L2 hit rate for one deep-K shape and then says such cases are rare. But the abstract claims L2 predictions are \"within roughly one percentage point\" on every GPU, which is clearly not true for that example. The real problem is that the paper never quantifies how many of the 703 shapes fall in the deep-K regime, what the latency MAPE is for those shapes, or how the optimism propagates into the pooled number. Since large-K GEMMs are exactly what LLM serving uses, this matters. It is a soft spot, but a fixable one: characterize the desync, add a correction factor, or restrict the claim.\n\nOther concerns are more conventional: no code or data released, so the empirical claims are not auditable; the GEMM evaluation filters out stream-K and SIMT fallback paths, narrowing the headline; and there is no run-to-run variance reporting. None of these are fatal. The baselines are handled fairly on the whole, and the distributed and end-to-end numbers appear solid.\n\nWho should read this: anyone building analytical cost models for tile-based DSLs, autotuning systems, or GPU kernel optimization. The paper gives a clear blueprint and a strong evaluation template.\n\nMy recommendation: send it to peer review with a request for artifact release and a deeper analysis of the deep-K bias. The central argument holds up, but the paper overstates the cache-accuracy claim in the abstract and needs to either fix the model or scope the claim. A serious referee will get value out of this.","headline":"A genuinely useful first-principles tile-level GPU cost model with unusually broad validation; the deep-K cache optimism is real but looks like an addressable limitation, not a fatal flaw.","tokens_in":25174,"tokens_out":1654,"would_cite":true,"duration_ms":22481,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training-free tile-centric model predicts GPU kernel latency to within about 12 percent and cache hit rates to within one point across four GPU generations.","keywords":["GPU performance modeling","analytical performance model","tile-centric programming","cache reuse distance","pipeline overlap","tensor-core GEMM","distributed inference","LLM serving"],"falsifier":"Run a suite of high-occupancy GEMMs with very large K, for example K in the tens of thousands, on a modern GPU and compare predicted versus measured L2 hit rate and latency. If hit-rate predictions are systematically optimistic in this regime, as the paper's own 82%-versus-43% example suggests, then the uniform-SM premise fails precisely for the workloads that dominate large-batch AI serving, and the central latency claim should be scoped accordingly.","tokens_in":24000,"feed_emoji":"⚡","tokens_out":5584,"duration_ms":63584,"temperature":0.7,"pith_summary":"TileSight tries to establish that the tile — the same unit modern GPU kernels are written in — can also be the unit of performance analysis, making kernel latency, cache behavior, and distributed communication fall out of one analytical engine. Its central claim is that with only one-time microbenchmark calibration per GPU, it predicts 703 tensor-core GEMM shapes across A100, H200, B200, and B6000 to a pooled mean absolute percentage error of 12.35%, beating prior learned and roofline-style predictors, and keeps L2 cache hit rates within about one percentage point. It extends the same tile abstraction to multi-GPU fused kernels and end-to-end LLM serving, reaching 16.18% and 13.52% weighted MAPE. A sympathetic reader would care because tile-centric programming is now the main way people write high-performance AI kernels, and existing tools are either coarse bounds, opaque learned models, or post-hoc profilers. The paper flags one regime where its uniform-SM assumption is optimistic — deep-K GEMMs — so the overall claim comes with that caveat.","feed_headline":"No-training model predicts GPU kernel speed within 12%","feed_subtitle":"Tile-centric simulator also gets L2 cache-hit rates to within one point and scales from one GPU to 32-GPU LLM serving.","key_machinery":"The central machinery is the tile execution plan: each tile carries a resource vector, a vector of times on independently schedulable hardware pipelines; producer–consumer edges fix legal orderings; and a recursive pipeline envelope T = T_pro + max(N−d,0)T_steady + T_epi, with effective depth d = stages × resident tiles per SM − 1, models overlap. Cache behavior is derived from tile reuse distance — the number of distinct tile-sized blocks accessed between two uses of the same block — evaluated with a Gaussian approximation of the stochastic cache model, so L1.5/L2/DDR traffic fractions fall out of the schedule instead of trace simulation. Distributed movement enters the same vector as a net","core_discovery":"On the paper's own terms, the discovery is that a tile execution plan — a graph of tile actions annotated with per-tile resource vectors over tensor-core, CUDA-core, SFU, on-chip tensor memory, shared memory, L1.5, L2, DDR, and network pipelines — is a sufficient description of a kernel to predict how it will run. Tiles carry dependencies, concurrency, and execution order; a recursive prologue–steady–epilogue envelope computes overlap; a tile-granular reuse-distance analysis with a Gaussian approximation of the stochastic cache model derives multi-level cache hit rates from the schedule; and remote tensor accesses become tiles with a network resource entry routed through an alpha–beta stage","pith_inferences":["Because the optimistic deep-K bias comes from the uniform-SM-progress assumption, adding a model of SM desynchronization, such as per-wave jitter on reuse distances, should close the gap without abandoning the analytical approach.","The tile-granularity reuse-distance idea is not tied to one vendor; any accelerator with tile-shaped work and hierarchical caches could be calibrated the same way, though latency-bound small-batch kernels remain outside the current model.","A direct consequence the paper leaves implicit is that if this holds, learned predictors in compilers and schedulers could be replaced or bootstrapped by analytical models, reducing the cost and opacity of performance engineering for AI workloads.","The single-percentage-point L2 hit-rate claim suggests cache hierarchies can be modeled without detailed traces, which could make analytical models competitive with simulation at a fraction of the cost."],"forward_implications":["Kernel developers can evaluate tile shapes, swizzles, pipeline depths, and block counts analytically before running, replacing blind autotuning on the target machine.","The model transfers across GPU generations from one-time microbenchmarks, so new hardware needs only rate calibration, not re-training or corpus collection.","Cache hit-rate prediction within about one percentage point makes L2-locality optimizations such as traversal order and persistent blocks testable inside a cost model.","The same tile abstraction scales to multi-GPU fused kernels and end-to-end serving, so distributed scheduling and collective choice can be reasoned about before deployment.","Used as a compiler cost model, it can prune 95% of schedule candidates while retaining 99.66% of exhaustive-search best performance."],"fun_headline_variants":["Tile-centric model predicts GPU kernel speed within 12%","From tiles to clusters: analytical GPU performance model","No training needed: tile model hits 12% GPU latency error","Tile reuse distance predicts L2 hits within one point"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that all SMs advance through the tile grid in lockstep, so cache traffic fractions can be computed from a single reuse-distance sequence; the paper's own deep-K GEMM case, 82% predicted versus 43% measured L2 hit rate, shows this assumption gives way exactly when SMs desynchronize.","fun_headline_variants_meta":{"raw":{"variants":["Tile-centric model predicts GPU kernel speed within 12%","From tiles to clusters: analytical GPU performance model","No training needed: tile model hits 12% GPU latency error","Tile reuse distance predicts L2 hits within one point"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000215,"raw_usage":{"total_tokens":1346,"prompt_tokens":905,"completion_tokens":441,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":649,"completion_tokens_details":{"reasoning_tokens":375}},"tokens_in":649,"tokens_out":441,"duration_ms":5960,"temperature":1.0,"reasoning_tokens":375,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T04:44:24.272667+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a suite of high-occupancy GEMMs with very large K, for example K in the tens of thousands, on a modern GPU and compare predicted versus measured L2 hit rate and latency. If hit-rate predictions are systematically optimistic in this regime, as the paper's own 82%-versus-43% example suggests, then the uniform-SM premise fails precisely for the workloads that dominate large-batch AI serving, and the central latency claim should be scoped accordingly.","supporting_citations":[],"review_version":1}