Details of Experiments This section provides detailed configurations for both pre- training and finetuning experiments

11 One LR Doesn’t Fit All: Heavy-Tail Guided Layerwise Learning Rates for LLMs Appendix A · 2048