TokenWeave achieves up to 1.28x lower latency and 1.19x higher throughput for distributed LLM inference by enabling compute-communication overlap at small token counts via a fused AllReduce-RMSNorm kernel that uses only 2-8 SMs.
ISBN 9781450399159
7 Pith papers cite this work, alongside 13 external citations. Polarity classification is still indexing.
7
Pith papers citing it
13
external citations · external index
representative citing papers
CTA-pipelining reduces 2-layer GEMM latency up to 31.8% vs micro-batching and 29.6% vs Tensor Parallelism on 8-GPU H200/B200 systems by CTA-level cross-GPU pipelining.
Geo is a framework for optimizing graph pattern matching queries via rewrite rules and equality saturation that discovers equivalences and reduces costs by up to 99%.
TLX introduces MIMW-based extensions to Triton that let developers orchestrate warp-group execution and asynchronous hardware features while preserving blocked programming productivity, with kernels deployed in large-scale training and inference.
DITRON introduces a hierarchical multi-level tiling compiler for distributed tensor programs that matches or exceeds expert CUDA libraries with 6-30% speedups and has been deployed to improve training MFU by over 10% while saving hundreds of thousands of GPU hours monthly.
DMA offloads on AMD MI300X GPUs are extended to latency-bound ML communication using untapped hardware features, closing up to 4.5x performance gap versus RCCL in collectives and delivering up to 1.5x lower latency and 1.9x higher throughput in LLM inference over vLLM.
A method using shared-memory occupancy shaping and elevated communication priority achieves up to 25.5% faster multi-GPU ML execution on NVIDIA and AMD GPUs.
citing papers explorer
-
TokenWeave: Efficient Compute-Communication Overlap for Distributed LLM Inference
TokenWeave achieves up to 1.28x lower latency and 1.19x higher throughput for distributed LLM inference by enabling compute-communication overlap at small token counts via a fused AllReduce-RMSNorm kernel that uses only 2-8 SMs.
-
CTA-Pipelining: A Latency-Oriented Spatial Scaling Method for Multi-GPU Systems
CTA-pipelining reduces 2-layer GEMM latency up to 31.8% vs micro-batching and 29.6% vs Tensor Parallelism on 8-GPU H200/B200 systems by CTA-level cross-GPU pipelining.
-
Geo: A Query Rewrite Framework for Graph Pattern Mining
Geo is a framework for optimizing graph pattern matching queries via rewrite rules and equality saturation that discovers equivalences and reduces costs by up to 99%.
-
TLX: Hardware-Native, Evolvable MIMW GPU Compiler for Large-scale Production Environments
TLX introduces MIMW-based extensions to Triton that let developers orchestrate warp-group execution and asynchronous hardware features while preserving blocked programming productivity, with kernels deployed in large-scale training and inference.
-
DITRON: Distributed Multi-level Tiling Compiler for Parallel Tensor Programs
DITRON introduces a hierarchical multi-level tiling compiler for distributed tensor programs that matches or exceeds expert CUDA libraries with 6-30% speedups and has been deployed to improve training MFU by over 10% while saving hundreds of thousands of GPU hours monthly.
-
DMA-Latte: Expanding the Reach of DMA Offloads to Latency-bound ML Communication
DMA offloads on AMD MI300X GPUs are extended to latency-bound ML communication using untapped hardware features, closing up to 4.5x performance gap versus RCCL in collectives and delivering up to 1.5x lower latency and 1.9x higher throughput in LLM inference over vLLM.
-
Resource-aware Computation-Communication Overlap for multi-GPU ML Workloads
A method using shared-memory occupancy shaping and elevated communication priority achieves up to 25.5% faster multi-GPU ML execution on NVIDIA and AMD GPUs.