UnionSparse combines a shared-bitmap sparse format with a Tensor Core SpMM kernel to speed up low-bit sparse LLM decoding on edge GPUs by cutting index traffic.
Intelligent Assistant Language Understanding On Device
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
It has recently become feasible to run personal digital assistants on phones and other personal devices. In this paper we describe a design for a natural language understanding system that runs on device. In comparison to a server-based assistant, this system is more private, more reliable, faster, more expressive, and more accurate. We describe what led to key choices about architecture and technologies. For example, some approaches in the dialog systems literature are difficult to maintain over time in a deployment setting. We hope that sharing learnings from our practical experiences may help inform future work in the research community.
citation-role summary
background 1
citation-polarity summary
fields
cs.DC 1years
2026 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
UnionSparse: An Index-Efficient Sparsity Framework for Low-Bit Sparse LLM Inference on Edge
UnionSparse combines a shared-bitmap sparse format with a Tensor Core SpMM kernel to speed up low-bit sparse LLM decoding on edge GPUs by cutting index traffic.