A modular sparse-attention framework with CPU KV offloading and layer-wise RoPE adjustment runs Llama-class models on 3M-token contexts on a single 48GB GPU without training.
NTK - Aware Scaled RoPE allows LLaMA models to have extended (8k+) context size without any fine-tuning and minimal perplexity degradation., June 2023
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
baseline 1
citation-polarity summary
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1roles
baseline 1polarities
baseline 1representative citing papers
citing papers explorer
-
InfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPU
A modular sparse-attention framework with CPU KV offloading and layer-wise RoPE adjustment runs Llama-class models on 3M-token contexts on a single 48GB GPU without training.