Pruning-aware pretraining iteratively deletes low-saliency parameter groups from a larger pretrained LLM during continued training, yielding sub-1B EfficientLLM models that outperform direct-pretraining baselines.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
EfficientLLM: Scalable Pruning-Aware Pretraining for Architecture-Agnostic Edge Language Models
Pruning-aware pretraining iteratively deletes low-saliency parameter groups from a larger pretrained LLM during continued training, yielding sub-1B EfficientLLM models that outperform direct-pretraining baselines.