The authors introduce a compute-dependent versus compute-independent framework and report nanoGPT experiments showing compute-independent algorithms such as LayerNorm and RoPE give compute-equivalent gains up to 1.9x, while compute-dependent ones become neutral only as model size grows.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Rethinking LLM Advancement: Compute-Dependent and Independent Paths to Progress
The authors introduce a compute-dependent versus compute-independent framework and report nanoGPT experiments showing compute-independent algorithms such as LayerNorm and RoPE give compute-equivalent gains up to 1.9x, while compute-dependent ones become neutral only as model size grows.