REVIEW 3 major objections 4 minor 10 references
What Irregularity Costs: CUDA C++, Rust, and Triton on a Hash-Blocked GPU Workload
T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read This paper measures a hash-blocked TSDF fusion kernel in CUDA C++, Rust, and Triton and finds that on the irregular allocate stage Triton is 11.2–31.6 times slower than CUDA C++ while Rust stays within 1.02–3.34 times, tracing both gaps…
desk verdict Strong methodology and a credible Triton result, but the central Rust fast path is a data race—the paper's safe/unsafe boundary claim needs rework before it is cited. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is the allocate-stage insert protocol: an open-addressed hash table with 64-bit packed block coordinates, a compare-exchange that publishes the key, a two-step publication window in which the block index is stored after the key wins the slot, and an unbounded per-lane probe loop. That protocol exposes what Triton cannot say—a probe loop must run to a compile-time bound with no per-lane early exit, and its compare-exchange takes no mask, forcing a scratch buffer for already-resolved lanes—and what Rust's safe idiom costs: a GPU-scope atomic load must be coherent across streaming multiprocessors, and because no L1 cache on the GPUs measured is, the type-correct read bypasses L1 on every access. The same kernel's update stage, a regular walk-and-accumulate, supplies the control condition that isolates these mechanisms.
What would settle it
Decouple the hash table size from the voxel pool so load factor exceeds 0.283 and rerun the sweep; if Triton's allocate ratio does not fall toward CUDA C++'s as probe chains genuinely lengthen, the compile-time-bound mechanism is wrong. Separately, run the pre-fix Rust kernel on a GPU whose L1 is coherent across streaming multiprocessors; if the L1 hit-rate deficit and the slowdown do not appear, the coherence premise is false.
Extended reading notes
Core claim
The paper's central claim is that the cost of a GPU language depends on whether the kernel is regular or irregular. On the regular update stage all three languages land within a small factor of each other, while on the irregular allocate stage Triton's ratio to hand-written CUDA C++ ranges from 11.2 to 31.6 times and Rust's from 1.02 to 3.34 times. The Triton gap is attributed not to instruction counts or code generation but to two expressiveness limits: the probe loop's trip count must be a compile-time constant, and the compare-exchange cannot be masked, forcing a scratch structure with no counterpart in CUDA. The Rust gap is invisible in every instruction count—the Rust kernel issues fewer instructions, fewer compare-exchanges, and fewer registers at identical occupancy—and is located by hardware counters in L1 residency: a GPU-scope atomic load must be coherent across streaming multiprocessors, and no measured L1 cache is, so every such read bypasses the cache. The paper also establishes a correctness consequence: Triton's bounded probe silently discards blocks at load factors an ordinary depth trajectory reaches, losing whole patches of surface with nothing reported.
Load-bearing premise
The Rust slowdown is blamed on a hardware fact: on the GPUs measured, the L1 cache is not coherent across streaming multiprocessors, so a GPU-scope atomic load must bypass L1 on every access; if that fact were false on some architecture, the explanation of Rust's allocate-stage cost would collapse.
Editorial extensions
If this is right
- Benchmarks on tiled dense linear algebra understate real language costs: languages that are close on regular work separate by more than an order of magnitude on hash probing and contended scatter.
- A Triton implementation of an unbounded-probe hash structure must fix a compile-time probe bound; at load factors above about 0.07 that bound can silently drop blocks or contributions, so reconstruction consumers can get holes or slightly wrong surfaces with no diagnostic.
- Rust programmers on the GPUs measured pay an L1 miss for the type-correct way to read shared state; matching CUDA speed requires a plain load whose safety rests on an algorithmic argument the compiler cannot check.
- A mixed-toolchain pipeline is the natural remedy: the full integrate path with Rust stays within 1 to 3 percent of hand-written CUDA C++ on real data, while Triton totals are 2.6 to 2.9 times slower, and the entire gap concentrates in one stage a practitioner could write in another language.
- The scoped-atomic load/store defect in the Rust-to-PTX compiler has been fixed upstream, so the Rust measurements reflect the fixed toolchain rather than a workaround.
Reading between the lines
- An extension of the argument, which the paper does not measure, is that the same Triton limits should depress other irregular GPU workloads—hash join builders, sparse matrix assembly, dynamic work queues—since they share data-dependent probe depth and contended scatter.
- The paper's sharpest untested prediction is that Triton's allocate ratio should shrink as the hash table fills; a natural follow-up would decouple table size from pool capacity to reach load factors above 0.283, and a negative result there would undercut the mechanism.
- If a future GPU made L1 coherent across streaming multiprocessors, the Rust gap the paper attributes to the memory model would presumably vanish, making the ranking hardware-dependent even though the language design is not.
- Other GPU languages that offer a scoped atomic load as the idiomatic read of shared state likely inherit the same L1-bypass cost, so the 'safe construct is the expensive one' tension is a memory-model property rather than a Rust-specific quirk.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper measures a hash-blocked TSDF fusion kernel implemented in CUDA C++, Rust via NVIDIA's cuda-oxide, and Triton, across a workload matrix on two sm 120 GPUs. The central result is a stage-dependent split: on the regular update stage the three languages are within a small factor, while on the irregular allocate stage Triton is 11.2-31.6x slower than CUDA C++ and Rust is 1.02-3.34x slower. The paper attributes the Triton gap to the absence of per-lane early exit and to maskless tl.atomic_cas, and the Rust gap to GPU-scope atomic loads bypassing L1; it also reports a correctness failure for bounded Triton probes at realistic load factors, a cuda-oxide defect that was fixed upstream, and a detailed measurement methodology with several documented failure modes.
Significance. This is a valuable and unusually disciplined measurement study. It targets a genuine gap in the GPU-language comparison literature by choosing an irregular, data-dependent workload rather than tiled dense linear algebra. The methodology is exemplary in several respects: correctness gates ordered by what they can catch, interleaved and rotated arm order, exclusive-device checks, distribution reporting rather than bare means, and symmetric re-runs that caught earlier wrong conclusions. The attribution effort is also strong: the Triton explanation is supported by varying the probe bound and by a control replacing the CAS with a plain load, and the Rust explanation is backed by hardware counters rather than by instruction counts. The paper honestly reports its own earlier errors, the untested load-factor prediction, the unattributed Rust residual, and the single-architecture limitation. The release of implementations, raw CSVs, and table-generation scripts is a real strength. If the Rust fast-path issue identified below is resolved, the paper's qualitative claim about Triton is convincing and the paper would be a solid contribution.
major comments (3)
- [Section 6.3 and Table 4] The claim 'This is correct rather than a relaxation of correctness' is not a memory-model argument. The 'Rust, fixed' row replaces scoped atomic loads of the probe key and published block index with plain loads while other threads may concurrently write those locations through atomicCAS and atomicExch. Under the Rust memory model this is a data race and undefined behavior; the algorithmic observation that a stale key costs one extra probe and a stale index only enters the wait loop does not make the access race-free. Consequently Table 4's 'Rust, fixed' row is not a type-correct Rust program; it is an unsafe CUDA-style idiom equivalent to what the CUDA C++ arm does. Because the fast Rust numbers are used to support the abstract's claim that Rust stays close to CUDA after correcting 'a single idiom,' the manuscript must be revised to state explicitly that the fast path is unsafe, or to present the result as 'safe Rust pays the L1-bypass cost; unsafe raw-pointer reads match CUDA.' This also affects Section 7.4's conclusion that 'the type-correct way to read shared state is the expensive one,' which is only correct if 'type-correct' means 'safe by the type system' and the fast path is acknowledged as unsafe.
- [Section 7.4] The sentence 'both constructs Rust offers for reading a location another thread writes bypass the cache' is inaccurate as stated: raw-pointer plain reads, which the Section 6.3 fix must use, are a third construct and are exactly the ones that do not bypass L1. The claim should be restricted to safe or type-checked constructs. This is not merely a wording issue: it changes the nature of the Rust result. The finding is that safe Rust cannot express the fast idiom without moving to unsafe code, a safe/unsafe boundary, rather than that every Rust construct for shared reads is expensive. The paper should say so explicitly and re-label the 'Rust, fixed' variant accordingly.
- [Section 8 and Section 6.3] The Rust attribution rests on the hardware premise that a GPU-scope atomic load must bypass L1 because no NVIDIA L1 cache is coherent across streaming multiprocessors. All measurements are on sm 120, and the statement that 'this holds on every current architecture' is asserted without a citation or a measurement on a different architecture. The paper already lists 'One architecture' as a threat to validity, but because this premise is the mechanism that explains the Rust slowdown, the manuscript should either cite an authoritative source (for example, the PTX ISA documentation or an NVIDIA architecture whitepaper) or explicitly narrow the generalization to the Blackwell architecture. As written, the reader cannot tell whether this is a measured fact or an assumed hardware property.
minor comments (4)
- [Abstract and Table 4] The abstract says the Rust kernel 'issues fewer instructions,' but Table 4 reports static SASS instructions of 520 for CUDA C++ and 528 for Rust. The body text more carefully says fewer branches and fewer global memory operations; the abstract should be corrected to match Table 4.
- [Section 7.2] The table of load factors and lost blocks has no header row and no explicit statement of the Triton probe bound used. Please add a header and state MAX_PROBE for that table, since the correctness discussion depends on it.
- [Section 5.1] In Figure 2's caption and text, ranges such as '1.12 2.56×' would be clearer as '1.12-2.56×' with consistent spacing; the current formatting is easy to misread as two separate numbers.
- [Section 6.4] The sentence '1.64× becomes 1.71× at 320k points and 1.53× becomes 1.39× at 1.28M' is hard to parse because it does not say which ratio is before and which is after, and for which arm. Please clarify.
Circularity Check
No circularity: the load-bearing claims are measurements with experimentally tested mechanisms, and the sharpest prediction is explicitly left untested rather than fitted.
full rationale
This is a measurement study, not a derivation chain, and its load-bearing claims are supported by controls rather than by fitted inputs or self-citation. The Triton attribution (bounded probe loop and unmasked tl.atomic_cas) is tested by varying the probe bound, which produces a linear cost (8.38x for an 8x increase), and by a control that replaces the compare-exchange with a plain load, which collapses 2.04 ms to 0.090 ms (Section 6.1). The Rust attribution (GPU-scope atomic loads bypassing L1) is tested by hardware counters — L1 hit rate 28.9% vs 55.9%, 1.70x L2 sectors — and by the two-line plain-load fix that restores every counter to parity (Section 6.3, Table 4, Figure 4). The paper's sharpest prediction (Triton's ratio shrinking as the table fills) is explicitly reported as untested because the reachable load factor caps at 0.283 (Sections 5.2 and 8); an untested prediction is the opposite of a fitted input. The only self-citation is the author's merged cuda-oxide pull request [3], cited for the defect fix; that claim is externally verifiable (merged upstream with tests and CI registration) and is not load-bearing for the performance attributions. The paper proactively removes the one design element it calls 'weakly circular' — certifying arms against a vendored reference implementation (Section 3.2) — and replaces it with closed-form geometry. The 'circularity' discussed in Section 2.3 is an algorithmic dependency (a block's location is known only after winning the slot), not a reasoning circularity, and it is disclosed and addressed in Section 6.4. Residual Rust costs are reported as unattributed rather than assigned to a convenient cause. A possible data-race objection to the 'Rust, fixed' kernel is a correctness or validity concern, not a circularity, since the measurements are what they are regardless of whether the idiom is type-correct under Rust's memory model.
Assumptions & free parameters
free parameters (3)
- Triton MAX_PROBE compile-time bound =
8 (sweep) and 32 in tests
- Hash table sizing factor (2x pool) =
table = 2 * pool capacity
- Triton scratch region indexing =
per-program (corrected from per-lane)
assumptions (4)
- domain assumption No NVIDIA L1 cache is coherent across streaming multiprocessors.
- domain assumption Triton's tl.static_range trip count is a compile-time constant and the loop has no per-lane early exit.
- domain assumption tl.atomic_cas has no mask parameter, unlike tl.atomic_add.
- domain assumption The hash-blocked TSDF workload is representative of a wider class of irregular GPU workloads (hash tables, sparse builders, work queues).
Cite this review
Pith. "Pith review of What Irregularity Costs: CUDA C++, Rust, and Triton on a Hash-Blocked GPU Workload." pith.science (2026). https://pith.science/paper/YBEZPEUZ
@misc{pith2026260808287,
author = {Pith},
title = {Pith review of: What Irregularity Costs: CUDA C++, Rust, and Triton on a Hash-Blocked GPU Workload},
year = {2026},
howpublished = {\url{https://pith.science/paper/YBEZPEUZ}},
note = {Machine review of arXiv:2608.08287}
}
read the original abstract
GPU language comparisons are almost always run on tiled dense linear algebra, where every toolchain is good and the differences are small. We implement the same hash-blocked TSDF fusion kernel in CUDA C++, in Rust through NVIDIA's cuda-oxide, and in Triton, and measure it on a workload with the opposite character: an open-addressed hash table with compare-exchange insertion, data-dependent per-lane probe depth, and contended scatter. The result is a split. On the regular stage, which walks a truncation band and accumulates, all three languages land within a small factor of each other. On the irregular stage, which probes and inserts, Rust stays close to hand-written CUDA C++ while Triton is more than an order of magnitude slower. Language choice is nearly free on the work that is usually benchmarked and expensive on the work that is not. We attribute both gaps to specific things the languages cannot express, not to ratios. Triton's cost follows from a probe loop that must run to a compile-time bound and from tl.atomic_cas taking no mask, which forces a scratch structure with no counterpart in CUDA. Rust's cost was invisible in every instruction count: its kernel issues fewer instructions, fewer compare-exchanges and fewer registers at identical occupancy, yet was slower. Hardware counters located it in L1 residency. A GPU-scope atomic load must be coherent across SMs, no NVIDIA L1 is, so the type-correct way to read a shared location bypasses the cache on every access. Triton's bounded probe is also a correctness problem for fusion: at load factors an ordinary depth trajectory reaches, it silently discards blocks and the reconstruction loses patches of surface with nothing reported. We also report a defect found and fixed in cuda-oxide itself, now merged upstream: its scoped atomic load and store could not be called at all in the build mode that produces real kernels.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
A volumetric method for building complex models from range images
Brian Curless and Marc Levoy. A volumetric method for building complex models from range images. InProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), pages 303–312, 1996
work page 1996
-
[2]
Joshua H. Davis, Pranav Sivaraman, Joy Kitson, Konstantinos Parasyris, Harshitha Menon, Isaac Minn, Giorgis Georgakoudis, and Abhinav Bhatele. Taking GPU programming models to task for performance portability.arXiv preprint arXiv:2402.08950, 2024. Conference version: https://doi.org/10.1145/3721145.3730423
arXiv 2024
-
[3]
fix(mir-lower): lower scoped atomic load/store and fences to inline PTX
Petr Korolev and cuda-oxide maintainers. fix(mir-lower): lower scoped atomic load/store and fences to inline PTX. NVlabs/cuda-oxide pull request #695, merged asc68632a3, 2026
work page 2026
-
[4]
Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J
Richard A. Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J. Davison, Pushmeet Kohli, Jamie Shotton, Steve Hodges, and Andrew Fitzgibbon. KinectFusion: Real-time dense surface mapping and tracking. In10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 127–136, 2011. 22
work page 2011
-
[5]
Real-time 3D reconstruction at scale using voxel hashing.ACM Transactions on Graphics, 32(6), 2013
Matthias Nießner, Michael Zollh¨ ofer, Shahram Izadi, and Marc Stamminger. Real-time 3D reconstruction at scale using voxel hashing.ACM Transactions on Graphics, 32(6), 2013
work page 2013
-
[6]
cuda-oxide: A Rust to PTX compiler
NVIDIA. cuda-oxide: A Rust to PTX compiler. https://github.com/NVlabs/cuda-oxide,
-
[7]
Philippe Tillet, H. T. Kung, and David Cox. Triton: An intermediate language and compiler for tiled neural network computations. InProceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages (MAPL), 2019
2019
-
[8]
TartanAir: A dataset to push the limits of visual SLAM
Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. TartanAir: A dataset to push the limits of visual SLAM. InIEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020
work page 2020
Show all 10 references
-
[9]
Open3D: A modern library for 3D data processing.arXiv preprint arXiv:1801.09847, 2018
Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun. Open3D: A modern library for 3D data processing.arXiv preprint arXiv:1801.09847, 2018. 23
2018 arXiv
-
[2026]
Measured at commit2db9713
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.