AQPIM performs in-memory product quantization of activations for LLMs on PIM hardware, reducing GPU-CPU communication by 90-98.5% and delivering 3.4x speedup over prior PIM methods.
Squeezed attention: Accelerat- ing long context length llm inference
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
verdicts
UNVERDICTED 3roles
background 1polarities
background 1representative citing papers
OmniMem achieves 2-4% higher accuracy than training-free baselines on long video benchmarks for audio-visual LLMs by using modality-aware KV cache allocation and perturbation-aware state selection, with further gains from budget-aware fine-tuning.
STARC remaps sparse KV caches by semantic clustering for PIM hardware, delivering 19-31% lower attention latency and 19-27% lower energy versus token-wise sparsity, with larger gains under tight KV budgets.
citing papers explorer
-
AQPIM: Breaking the PIM Capacity Wall for LLMs with In-Memory Activation Quantization
AQPIM performs in-memory product quantization of activations for LLMs on PIM hardware, reducing GPU-CPU communication by 90-98.5% and delivering 3.4x speedup over prior PIM methods.
-
OmniMem: Perturbation-aware Memory Compression for Streaming Audio-Visual LLMs
OmniMem achieves 2-4% higher accuracy than training-free baselines on long video benchmarks for audio-visual LLMs by using modality-aware KV cache allocation and perturbation-aware state selection, with further gains from budget-aware fine-tuning.
-
Sparse Attention Remapping with Clustering for Efficient LLM Decoding on PIM
STARC remaps sparse KV caches by semantic clustering for PIM hardware, delivering 19-31% lower attention latency and 19-27% lower energy versus token-wise sparsity, with larger gains under tight KV budgets.