REVIEW 3 cited by
The Case for Co-Designing Model Architectures with Hardware
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
While GPUs are responsible for training the vast majority of state-of-the-art deep learning models, the implications of their architecture are often overlooked when designing new deep learning (DL) models. As a consequence, modifying a DL model to be more amenable to the target hardware can significantly improve the runtime performance of DL training and inference. In this paper, we provide a set of guidelines for users to maximize the runtime performance of their transformer models. These guidelines have been created by carefully considering the impact of various model hyperparameters controlling model shape on the efficiency of the underlying computation kernels executed on the GPU. We find the throughput of models with efficient model shapes is up to 39\% higher while preserving accuracy compared to models with a similar number of parameters but with unoptimized shapes.
Forward citations
Cited by 3 Pith papers
-
Arctic Long Sequence Training: Scalable And Efficient Training For Multi-Million Token Sequences
ALST combines tiled loss and MLP compute, Ulysses-style sequence parallelism for Hugging Face GQA and MQA models, and activation-checkpoint CPU offload to train Llama-8B at up to 500K tokens on one H100 and 15M tokens...
-
The Zamba2 Suite: Technical Report
This paper introduces Zamba2, a suite of 1.2B, 2.7B, and 7.4B hybrid Mamba2-transformer models that claims state-of-the-art small-model quality and 30-50% lower time-to-first-token, with open weights and a 5T-token pr...
-
Best Practices for Large Language Models in Radiology
The paper recommends starting LLM use in radiology with prompt optimization and retrieval augmentation, fine-tuning only when needed, and preferring locally hosted open models with human expert evaluation.
Discussion (0). Continue with ORCID to comment.