A unified HPO pipeline built on Optuna and Ray Tune is applied to six time series forecasting models across three datasets, providing empirical guidance on hyperparameter choices.
Training Tips for the Transformer Model
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
This article describes our experiments in neural machine translation using the recent Tensor2Tensor framework and the Transformer sequence-to-sequence model (Vaswani et al., 2017). We examine some of the critical parameters that affect the final translation quality, memory usage, training stability and training time, concluding each experiment with a set of recommendations for fellow researchers. In addition to confirming the general mantra "more data and larger models", we address scaling to multiple GPUs and provide practical tips for improved training regarding batch size, learning rate, warmup steps, maximum sentence length and checkpoint averaging. We hope that our observations will allow others to get better results given their particular hardware and data constraints.
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
A Unified Hyperparameter Optimization Pipeline for Transformer-Based Time Series Forecasting Models
A unified HPO pipeline built on Optuna and Ray Tune is applied to six time series forecasting models across three datasets, providing empirical guidance on hyperparameter choices.