NNTile trains a custom 49.9-billion-parameter GPT-2-style model on a single 8-GPU node, versus about 10.6 billion with PyTorch FSDP.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
NNTile: a machine learning framework capable of training extremely large GPT language models on a single node
NNTile trains a custom 49.9-billion-parameter GPT-2-style model on a single 8-GPU node, versus about 10.6 billion with PyTorch FSDP.