Paper Citation Record · LEDGER

Benchmarking Optimizers for Large Language Model Pretraining

As of 10 August 2026, this Paper Citation Record lists 100 of 176 outbound references and 28 inbound Pith citation observations for arXiv:2509.01440.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2509.01440 v1

Coverage vector

measured 100 of 176 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-05T12:35:51.957442Z

measured 128 of 128 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-10T06:31:04.303077+00:00

measured 28 of 28 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-05T11:59:49.778357Z

measured 1 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-08-05T02:28:24.338817Z

Reference resolution

100 of 176 outbound references displayed

verified exact0
verified fuzzy3
unresolved97
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

0
arxiv_reference, observed 2026-08-05T02:28:24.338817Z

Outbound references

Observation f1e3d2e4-6b08-434e-ad32-7d90ef11da41 · outbound

This paper cites Dion: A communication-efficient optimizer for large models, 2025.

Benchmarking Optimizers for Large Language Model Pretraining Dion: A communication-efficient optimizer for large models, 2025

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.343838Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 63753a59-3c59-475d-be3d-de886728aeb2 · outbound

This paper cites an unresolved cited work.

Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.353802Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2d976c51-744f-4ac0-8423-f981979703e6 · outbound

This paper cites an unresolved cited work.

Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.376510Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3500af33-a6ce-4821-8c65-c2c3dc857f52 · outbound

This paper cites ASGO: Adaptive structured gradient optimization, 2025.

Benchmarking Optimizers for Large Language Model Pretraining ASGO: Adaptive structured gradient optimization, 2025

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.385438Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8ad23d38-276b-4fa0-b950-4e69775ba4f7 · outbound

This paper cites Dissecting adam: The sign, magnitude and variance of stochastic gradients, 2020.

Benchmarking Optimizers for Large Language Model Pretraining Dissecting adam: The sign, magnitude and variance of stochastic gradients, 2020

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.396648Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d65bb383-339a-4cbf-9f54-5dfddbcd63ef · outbound

This paper cites Bekas, E.

Benchmarking Optimizers for Large Language Model Pretraining Bekas, E

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.411974Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 99653034-b5f6-463c-8be3-ad2118a1de4c · outbound

This paper cites Straight to zero: Why linearly decaying the learning rate to zero works best for llms, 2025.

Benchmarking Optimizers for Large Language Model Pretraining Straight to zero: Why linearly decaying the learning rate to zero works best for llms, 2025

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.430977Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6cb39b0b-ccc6-4f97-8da4-694e1ceb2c73 · outbound

This paper cites Old optimizer, new norm: An anthology, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Old optimizer, new norm: An anthology, 2024

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.443584Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b232b8d0-7608-44ad-bd59-65bcca0fa84b · outbound

This paper cites signsgd: Compressed optimisation for non-convex problems, 2018.

Benchmarking Optimizers for Large Language Model Pretraining signsgd: Compressed optimisation for non-convex problems, 2018

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.461957Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9da88b8d-b881-4c84-ab66-b87e03d91e37 · outbound

This paper cites Pythia: A suite for analyzing large language models across training and scaling, 2023.

Benchmarking Optimizers for Large Language Model Pretraining Pythia: A suite for analyzing large language models across training and scaling, 2023

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.470832Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ca24bbd4-073f-45d3-8332-faf66c88c7b5 · outbound

This paper cites Prince, Björn Deiseroth, Andres Felipe Cruz-Salinas, Carlo Luschi, Samuel Weinbach, and Douglas Orr.

Benchmarking Optimizers for Large Language Model Pretraining Prince, Björn Deiseroth, Andres Felipe Cruz-Salinas, Carlo Luschi, Samuel Weinbach, and Douglas Orr

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.480601Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e8acd836-9d96-4b0f-b589-3a95b51d7fe0 · outbound

This paper cites Loss-to- loss prediction: Scaling laws for all datasets, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Loss-to- loss prediction: Scaling laws for all datasets, 2024

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.488446Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 48f85189-d84d-4b0b-8315-30ee869344d0 · outbound

This paper cites an unresolved cited work.

Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.496417Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e99a0e4f-801e-4090-8542-f186ea656dfa · outbound

This paper cites How to scale your ema, 2023.

Benchmarking Optimizers for Large Language Model Pretraining How to scale your ema, 2023

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.503429Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation edcf4437-997a-4819-8877-53086dc35559 · outbound

This paper cites Preconditioned spectral descent for deep learning.

Benchmarking Optimizers for Large Language Model Pretraining Preconditioned spectral descent for deep learning

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.517235Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation bfc187f7-6eb5-4361-bf37-c31eec335328 · outbound

This paper cites Communication-efficient language model training scales reliably and robustly: Scaling laws for diloco, 2025.

Benchmarking Optimizers for Large Language Model Pretraining Communication-efficient language model training scales reliably and robustly: Scaling laws for diloco, 2025

Reference 16

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.525690Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b635442a-42eb-4898-9d20-ebca781a4a6d · outbound

This paper cites an unresolved cited work.

Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.536046Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0218fbe8-da25-4d79-9dd4-28bcb6e12e43 · outbound

This paper cites Gradient clipping improves adagrad when the noise is heavy-tailed, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Gradient clipping improves adagrad when the noise is heavy-tailed, 2024

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.545978Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d15f6470-9b39-4ca2-8dd7-05ff919ba841 · outbound

This paper cites Palm: Scaling language modeling with pathways, 2022.

Benchmarking Optimizers for Large Language Model Pretraining Palm: Scaling language modeling with pathways, 2022

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.555425Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f8d737d2-f2e1-4aa1-b0dd-5c0eaa05a356 · outbound

This paper cites an unresolved cited work.

Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.580817Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 692e9966-eb88-4cf5-be99-1027a0932bb4 · outbound

This paper cites Why do we need weight decay in modern deep learning?, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Why do we need weight decay in modern deep learning?, 2024

Reference 21

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.590632Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation aa651e1c-ed7c-4aa4-9046-acbf225be9e2 · outbound

This paper cites Fu, Stefano Ermon, Atri Rudra, and Christopher Ré.

Benchmarking Optimizers for Large Language Model Pretraining Fu, Stefano Ermon, Atri Rudra, and Christopher Ré

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.599608Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3084708e-ad23-42b8-9fc3-7920679b9b97 · outbound

This paper cites Deepseek llm: Scaling open-source language models with longtermism, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Deepseek llm: Scaling open-source language models with longtermism, 2024

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.606542Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2d7c4d5d-fbb7-417a-b303-b8b142ed5d9b · outbound

This paper cites Deepseek-v3 technical report, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Deepseek-v3 technical report, 2024

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.618917Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7fe2fc43-b927-4b77-8a4a-7c57bd258c9f · outbound

This paper cites Why gradients rapidly increase near the end of training, 2025.

Benchmarking Optimizers for Large Language Model Pretraining Why gradients rapidly increase near the end of training, 2025

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.625839Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e0a9cbd7-3d3d-43b0-ab96-743e009cb0ed · outbound

This paper cites Optimal linear decay learning rate schedules and further refinements, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Optimal linear decay learning rate schedules and further refinements, 2024

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.637129Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0bd0961f-59d0-4736-8b53-fa3953b2643a · outbound

This paper cites The road less scheduled, 2024.

Benchmarking Optimizers for Large Language Model Pretraining The road less scheduled, 2024

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.646543Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0e9511fb-9e28-475e-acbb-6f7b128a5800 · outbound

This paper cites Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.

Benchmarking Optimizers for Large Language Model Pretraining Bert: Pre-training of deep bidirectional transformers for language understanding, 2019

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.653793Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c586caa8-8625-4451-9055-b8918360c51c · outbound

This paper cites The practitioner’s guide to the maximal update parameterization.

Benchmarking Optimizers for Large Language Model Pretraining The practitioner’s guide to the maximal update parameterization

Reference 29

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.660661Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b33a0093-52e9-41f7-aefe-ae400211107a · outbound

This paper cites Incorporating Nesterov Momentum into Adam, 2016.

Benchmarking Optimizers for Large Language Model Pretraining Incorporating Nesterov Momentum into Adam, 2016

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.667043Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3268a32e-81bb-4e54-a09f-a83543bb1881 · outbound

This paper cites Understanding emergent abilities of language models from the loss perspective, 2025.

Benchmarking Optimizers for Large Language Model Pretraining Understanding emergent abilities of language models from the loss perspective, 2025

Reference 31

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.675765Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 796c91ff-7051-4894-9d7e-a330a9565ede · outbound

This paper cites Adaptive subgradient methods for online learning and stochastic optimization.

Benchmarking Optimizers for Large Language Model Pretraining Adaptive subgradient methods for online learning and stochastic optimization

Reference 32

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.684220Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a236d56d-8584-43e9-8502-d6f3fba362c5 · outbound

This paper cites A simple convergence proof of adam and adagrad, 2022.

Benchmarking Optimizers for Large Language Model Pretraining A simple convergence proof of adam and adagrad, 2022

Reference 33

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.692572Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 78e07626-7a34-4416-924a-e3bf94d4a431 · outbound

This paper cites Data movement bottlenecks to large-scale model training: Scaling past 1e28 flop,.

Benchmarking Optimizers for Large Language Model Pretraining Data movement bottlenecks to large-scale model training: Scaling past 1e28 flop,

Reference 34

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.699221Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 59524814-5713-4ec8-b181-b2e0eabbc3b5 · outbound

This paper cites Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity, 2022.

Benchmarking Optimizers for Large Language Model Pretraining Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity, 2022

Reference 35

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.716164Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2772b691-3c1e-4765-86fb-a844a891ac4e · outbound

This paper cites A stable whitening optimizer for efficient neural network training, 2025.

Benchmarking Optimizers for Large Language Model Pretraining A stable whitening optimizer for efficient neural network training, 2025

Reference 36

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.728738Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1b9f62ea-dde8-4235-878b-87c7d31b8cbb · outbound

This paper cites Dimakis, Gabriel Ilharco, Pang Wei Koh, Shuran Song, Thomas Kollar, Yair Carmon, Achal Dave, Reinhard Heckel, Niklas Muennighoff, and Ludwig Schmidt.

Benchmarking Optimizers for Large Language Model Pretraining Dimakis, Gabriel Ilharco, Pang Wei Koh, Shuran Song, Thomas Kollar, Yair Carmon, Achal Dave, Reinhard Heckel, Niklas Muennighoff, and Ludwig Schmidt

Reference 37

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.740367Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 835f6f7a-b65f-4488-bf9a-02c82ab72dad · outbound

This paper cites The Pile: An 800GB Dataset of Diverse Text for Language Modeling.

Benchmarking Optimizers for Large Language Model Pretraining The Pile: An 800GB Dataset of Diverse Text for Language Modeling

Reference 38

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.748594Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 80ad567a-7f6b-4aff-99e4-4ac75f04e1de · outbound

This paper cites The pile: An 800gb dataset of diverse text for language modeling, 2020.

Benchmarking Optimizers for Large Language Model Pretraining The pile: An 800gb dataset of diverse text for language modeling, 2020

Reference 39

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.766003Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8b0d714c-c4f9-47a8-b0b7-36258def0546 · outbound

This paper cites Gemini: A family of highly capable multimodal models, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Gemini: A family of highly capable multimodal models, 2024

Reference 40

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.776544Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2081e7d5-38c0-4050-b7a7-cfbe7b09e64b · outbound

This paper cites A loss curvature perspective on training instability in deep learning, 2021.

Benchmarking Optimizers for Large Language Model Pretraining A loss curvature perspective on training instability in deep learning, 2021

Reference 41

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.786424Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4af57d0a-2b59-4138-96a3-72aaddd8365a · outbound

This paper cites A minimalist optimizer design for llm pretraining, 2025.

Benchmarking Optimizers for Large Language Model Pretraining A minimalist optimizer design for llm pretraining, 2025

Reference 42

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.793612Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0d192b09-2fc4-408a-b2fe-3a1ab03dd7f5 · outbound

This paper cites Generating sequences with recurrent neural networks, 2014.

Benchmarking Optimizers for Large Language Model Pretraining Generating sequences with recurrent neural networks, 2014

Reference 43

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.802389Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9d8e5927-11b3-4eec-b73c-53b342eb21fe · outbound

This paper cites Accelerating newton-schulz iteration for orthogonalization via chebyshev-type polynomials, 2025.

Benchmarking Optimizers for Large Language Model Pretraining Accelerating newton-schulz iteration for orthogonalization via chebyshev-type polynomials, 2025

Reference 44

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.816714Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 353c38e4-9f8c-496c-83b4-0664eed62531 · outbound

This paper cites AdaPlus: Integrating nesterov momentum and precise stepsize adjustment on adamw basis, 2023.

Benchmarking Optimizers for Large Language Model Pretraining AdaPlus: Integrating nesterov momentum and precise stepsize adjustment on adamw basis, 2023

Reference 45

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.825415Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation abceb8a5-5972-4481-b633-7da75869a1a4 · outbound

This paper cites Shampoo: Preconditioned stochastic tensor optimization, 2018.

Benchmarking Optimizers for Large Language Model Pretraining Shampoo: Preconditioned stochastic tensor optimization, 2018

Reference 46

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.838173Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9a5d9988-1f52-4382-b27a-8af80cf7ebc2 · outbound

This paper cites Under- standing and minimising outlier features in neural network training, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Under- standing and minimising outlier features in neural network training, 2024

Reference 47

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.847579Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4e150506-03f9-42ca-a4f1-16b9938798fe · outbound

This paper cites Deep residual learning for image recognition, 2015.

Benchmarking Optimizers for Large Language Model Pretraining Deep residual learning for image recognition, 2015

Reference 48

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.856289Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation fa584f48-d9d3-408a-8ea6-1ac6408545d8 · outbound

This paper cites an unresolved cited work.

Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work

Reference 49

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.866960Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e80a794b-b7a4-48cf-a3e8-04fe3aeb54f1 · outbound

This paper cites Neural networks for machine learning, lecture 6e rmsprop: Divide the gradient by a running average of its recent magnitude, 2012.

Benchmarking Optimizers for Large Language Model Pretraining Neural networks for machine learning, lecture 6e rmsprop: Divide the gradient by a running average of its recent magnitude, 2012

Reference 50

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.875428Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b225316f-3228-4134-873c-f2faf6e4d95b · outbound

This paper cites Rae, Oriol Vinyals, and Laurent Sifre.

Benchmarking Optimizers for Large Language Model Pretraining Rae, Oriol Vinyals, and Laurent Sifre

Reference 51

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.882146Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 73adb6cd-73a3-40a1-9be7-78c6242c9fff · outbound

This paper cites Minicpm: Unveiling the potential of small language models with scalable training strategies, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Minicpm: Unveiling the potential of small language models with scalable training strategies, 2024

Reference 52

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.892741Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 80d4d8db-1738-4952-96b9-346906feed94 · outbound

This paper cites Improving transformer opti- mization through better initialization.

Benchmarking Optimizers for Large Language Model Pretraining Improving transformer opti- mization through better initialization

Reference 53

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.905305Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a99c433d-c413-4835-b1ba-2d4a94238b8f · outbound

This paper cites Scaling laws and compute-optimal training beyond fixed training durations, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Scaling laws and compute-optimal training beyond fixed training durations, 2024

Reference 54

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.919282Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f50c55e8-f38c-4482-9a71-0bc3784b1f17 · outbound

This paper cites Averaging weights leads to wider optima and better generalization, 2019.

Benchmarking Optimizers for Large Language Model Pretraining Averaging weights leads to wider optima and better generalization, 2019

Reference 55

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.926941Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 13415e75-749f-4321-949c-d87048939e02 · outbound

This paper cites Intellect-1 technical report, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Intellect-1 technical report, 2024

Reference 56

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.938288Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f9d73ab2-3f1d-43a7-b8df-4e343381632d · outbound

This paper cites an unresolved cited work.

Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work

Reference 57

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.946890Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d820ecaa-a9fa-4067-a3dc-f4840cd3c4de · outbound

This paper cites modded-nanogpt: Speedrunning the nanogpt baseline, 2024.

Benchmarking Optimizers for Large Language Model Pretraining modded-nanogpt: Speedrunning the nanogpt baseline, 2024

Reference 58

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.956877Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 38974e3d-c682-4d85-a70a-786b757ad391 · outbound

This paper cites Muon: An optimizer for hidden layers in neural networks, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Muon: An optimizer for hidden layers in neural networks, 2024

Reference 59

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.965261Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c16bd359-1f7d-46b2-be05-164a43f14643 · outbound

This paper cites an unresolved cited work.

Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work

Reference 60

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.981425Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 92f7ebe0-b223-439b-9169-aecb5f374815 · outbound

This paper cites Why warmup the learning rate? underlying mechanisms and improvements, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Why warmup the learning rate? underlying mechanisms and improvements, 2024

Reference 61

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.990386Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e13ef9ac-1d10-4655-becb-8e4788c544a7 · outbound

This paper cites Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei.

Benchmarking Optimizers for Large Language Model Pretraining Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei

Reference 62

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:50.998917Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8a7e8f82-4584-47c5-a4db-6e4e0a4007a0 · outbound

This paper cites Error feedback fixes signSGD and other gradient compression schemes.

Benchmarking Optimizers for Large Language Model Pretraining Error feedback fixes signSGD and other gradient compression schemes

Reference 63

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.007417Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 337c6213-9588-41a8-a388-309d8927b8cc · outbound

This paper cites an unresolved cited work.

Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work

Reference 64

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.016356Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b2aa18dd-7536-4971-b794-82731facaa9d · outbound

This paper cites an unresolved cited work.

Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work

Reference 65

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.023163Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8b0ea370-d994-49a5-8950-e8350bee3995 · outbound

This paper cites Kingma and Jimmy Ba.

Benchmarking Optimizers for Large Language Model Pretraining Kingma and Jimmy Ba

Reference 66

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.029750Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a3881c39-deb2-4d95-8888-0b1319dfe729 · outbound

This paper cites Sign operator for coping with heavy-tailed noise in non-convex optimization: High probability bounds under (l0,l 1)-smoothness, 2025.

Benchmarking Optimizers for Large Language Model Pretraining Sign operator for coping with heavy-tailed noise in non-convex optimization: High probability bounds under (l0,l 1)-smoothness, 2025

Reference 67

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.041783Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cc26a4d6-5197-4e15-b87b-02d26576d30b · outbound

This paper cites Analyzing & reducing the need for learning rate warmup in gpt training, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Analyzing & reducing the need for learning rate warmup in gpt training, 2024

Reference 68

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.048805Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9bdf0811-3ca2-4b48-9288-c3cd48b5710d · outbound

This paper cites Rotational equilibrium: How weight decay balances learning across neural networks, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Rotational equilibrium: How weight decay balances learning across neural networks, 2024

Reference 69

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.055398Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8398ef5d-ff5e-4bcd-8e36-29e4c59cceaf · outbound

This paper cites Understanding gradient orthogonalization for deep learning via non-euclidean trust-region optimization, 2025.

Benchmarking Optimizers for Large Language Model Pretraining Understanding gradient orthogonalization for deep learning via non-euclidean trust-region optimization, 2025

Reference 70

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.060600Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a4b82c0a-b051-4229-ba82-4f198e4c51e0 · outbound

This paper cites Spector, Blake Bordelon, Niklas Muennighoff, Mansheej Paul, Cengiz Pehlevan, Christopher Ré, and Aditi Raghunathan.

Benchmarking Optimizers for Large Language Model Pretraining Spector, Blake Bordelon, Niklas Muennighoff, Mansheej Paul, Cengiz Pehlevan, Christopher Ré, and Aditi Raghunathan

Reference 71

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.074620Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0910f1f0-0f14-4d1d-aba3-0f1bdc01d2a5 · outbound

This paper cites Why do machine learning optimizers that work, work? PhD thesis, University of British Columbia, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Why do machine learning optimizers that work, work? PhD thesis, University of British Columbia, 2024

Reference 72

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.085256Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6fa78a03-397b-4db9-bdb9-d6eac5e83b0a · outbound

This paper cites Noise is not the main factor behind the gap between sgd and adam on transformers, but sign descent might be, 2023.

Benchmarking Optimizers for Large Language Model Pretraining Noise is not the main factor behind the gap between sgd and adam on transformers, but sign descent might be, 2023

Reference 73

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.098766Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 20c8f9dd-e67f-48dd-a6b4-ece308824be8 · outbound

This paper cites Heavy- tailed class imbalance and why adam outperforms gradient descent on language models, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Heavy- tailed class imbalance and why adam outperforms gradient descent on language models, 2024

Reference 74

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.112981Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b0fd45ae-a0e1-46aa-b445-6e6e38e53444 · outbound

This paper cites Farseer: A refined scaling law in large language models, 2025.

Benchmarking Optimizers for Large Language Model Pretraining Farseer: A refined scaling law in large language models, 2025

Reference 75

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.120086Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9d666264-eeb7-4454-822e-29c3b35cf984 · outbound

This paper cites Predictable scale: Part i – optimal hyperparameter scaling law in large language model pretraining, 2025.

Benchmarking Optimizers for Large Language Model Pretraining Predictable scale: Part i – optimal hyperparameter scaling law in large language model pretraining, 2025

Reference 76

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.139881Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7299b8a9-a840-4b50-b3df-edbd8a503584 · outbound

This paper cites Datacomp-lm: In search of the next generation of training sets for language models.

Benchmarking Optimizers for Large Language Model Pretraining Datacomp-lm: In search of the next generation of training sets for language models

Reference 77

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.152542Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5da4ae20-002b-4624-b747-5b594098b239 · outbound

This paper cites Pytorch distributed: Experiences on accelerating data parallel training, 2020.

Benchmarking Optimizers for Large Language Model Pretraining Pytorch distributed: Experiences on accelerating data parallel training, 2020

Reference 78

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.163892Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1c2cf729-c7d3-43ec-8d55-bf9798f8b90f · outbound

This paper cites Sophia: A scalable stochastic second-order optimizer for language model pre-training, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Sophia: A scalable stochastic second-order optimizer for language model pre-training, 2024

Reference 79

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.172192Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4061f7f4-ef2f-4a11-80c1-fc94bb050377 · outbound

This paper cites Same pre-training loss, better downstream: Implicit bias matters for language models, 2022.

Benchmarking Optimizers for Large Language Model Pretraining Same pre-training loss, better downstream: Implicit bias matters for language models, 2022

Reference 80

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.180465Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation bc4135a7-2505-4875-900b-84de062d0af9 · outbound

This paper cites Muon is scalable for llm training, 2025.

Benchmarking Optimizers for Large Language Model Pretraining Muon is scalable for llm training, 2025

Reference 81

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.191462Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation daaa7974-2a32-4a0b-8a88-d7b69a33f651 · outbound

This paper cites The llama 3 herd of models, 2024.

Benchmarking Optimizers for Large Language Model Pretraining The llama 3 herd of models, 2024

Reference 82

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.201700Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a16d0fcb-fc91-4260-9bda-8d80a8687569 · outbound

This paper cites Sgdr: Stochastic gradient descent with warm restarts, 2017.

Benchmarking Optimizers for Large Language Model Pretraining Sgdr: Stochastic gradient descent with warm restarts, 2017

Reference 83

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.211642Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 22c52868-f616-4c9e-ab23-bd15809fe719 · outbound

This paper cites Decoupled weight decay regularization, 2019.

Benchmarking Optimizers for Large Language Model Pretraining Decoupled weight decay regularization, 2019

Reference 84

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.224611Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e5d8e998-92e5-4744-aa5a-a4c761ed6385 · outbound

This paper cites SW AN: SGD with normalization and whitening enables stateless llm training, 2025.

Benchmarking Optimizers for Large Language Model Pretraining SW AN: SGD with normalization and whitening enables stateless llm training, 2025

Reference 85

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.235207Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5dcd21a0-44d1-42aa-bd47-0b5060f0111c · outbound

This paper cites Small batch size training for language models: When vanilla sgd works, and why gradient accumulation is wasteful, 2025.

Benchmarking Optimizers for Large Language Model Pretraining Small batch size training for language models: When vanilla sgd works, and why gradient accumulation is wasteful, 2025

Reference 86

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.242809Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b3c415c9-f333-43d9-862f-75f9d37f68fb · outbound

This paper cites New insights and perspectives on the natural gradient method, 2020.

Benchmarking Optimizers for Large Language Model Pretraining New insights and perspectives on the natural gradient method, 2020

Reference 87

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.248543Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c9e0dec5-84c7-4103-aa90-4253ea486bb8 · outbound

This paper cites Effects of parameter norm growth during transformer training: Inductive bias from gradient descent, 2023.

Benchmarking Optimizers for Large Language Model Pretraining Effects of parameter norm growth during transformer training: Inductive bias from gradient descent, 2023

Reference 88

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.255443Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0c7b8a1c-08c2-4457-8bb3-486055d882cc · outbound

This paper cites Mixed precision training, 2018.

Benchmarking Optimizers for Large Language Model Pretraining Mixed precision training, 2018

Reference 89

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.261498Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8f1263d3-4151-4df4-bfb7-35b3d19913d9 · outbound

This paper cites Prodigy: An expeditiously adaptive parameter-free learner, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Prodigy: An expeditiously adaptive parameter-free learner, 2024

Reference 90

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.275664Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a421d4d2-fa90-4870-84d4-980d0a231179 · outbound

This paper cites Connections between schedule-free optimizers, ademamix, and accelerated sgd variants, 2025.

Benchmarking Optimizers for Large Language Model Pretraining Connections between schedule-free optimizers, ademamix, and accelerated sgd variants, 2025

Reference 91

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.358568Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 404f1cf9-6ae6-4461-87eb-25729cb3cdb2 · outbound

This paper cites Nemirovskii and Yu.E.

Benchmarking Optimizers for Large Language Model Pretraining Nemirovskii and Yu.E

Reference 92

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.419005Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c51e241d-6516-4e23-b331-d3ca9c0a86d0 · outbound

This paper cites Nesterov and V.

Benchmarking Optimizers for Large Language Model Pretraining Nesterov and V

Reference 93

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.472058Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3137ecfe-fb7e-417d-b06a-36dc41aae6f9 · outbound

This paper cites A method for unconstrained convex minimization problem with the rate of convergenceo(1/k2), 1983.

Benchmarking Optimizers for Large Language Model Pretraining A method for unconstrained convex minimization problem with the rate of convergenceo(1/k2), 1983

Reference 94

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.531481Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation fa5b3773-9138-402e-ada3-c120a8190b59 · outbound

This paper cites Gpt-4 technical report, 2024.

Benchmarking Optimizers for Large Language Model Pretraining Gpt-4 technical report, 2024

Reference 95

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.607895Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b5dec2b1-a411-4fd3-8b95-6396e8d915f3 · outbound

This paper cites Neural networks (maybe) evolved to make adam the best optimizer, 2020.

Benchmarking Optimizers for Large Language Model Pretraining Neural networks (maybe) evolved to make adam the best optimizer, 2020

Reference 96

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.676668Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a68501d0-3c31-42a5-a91b-d30c574888f2 · outbound

This paper cites Open problem: Parameter-free and scale-free online algorithms.

Benchmarking Optimizers for Large Language Model Pretraining Open problem: Parameter-free and scale-free online algorithms

Reference 97

Resolution: verified fuzzy; raw_fallback, observed 2026-08-05T12:36:02.794621Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation 568a1e7d-7e71-4a89-88cc-4b6afea2f98d · outbound

This paper cites In search of adam’s secret sauce, 2025.

Benchmarking Optimizers for Large Language Model Pretraining In search of adam’s secret sauce, 2025

Reference 98

Resolution: verified fuzzy; raw_fallback, observed 2026-08-05T12:36:02.768138Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation a8f6afa5-eb57-45d5-bda7-46bd96386e7a · outbound

This paper cites The ademamix optimizer: Better, faster, older, 2024.

Benchmarking Optimizers for Large Language Model Pretraining The ademamix optimizer: Better, faster, older, 2024

Reference 99

Resolution: verified fuzzy; raw_fallback, observed 2026-08-05T12:36:02.733539Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation f12878bb-536d-4b0f-960d-a3a7eb347cb3 · outbound

This paper cites Pytorch: An imperative style, high-performance deep learning library, 2019.

Benchmarking Optimizers for Large Language Model Pretraining Pytorch: An imperative style, high-performance deep learning library, 2019

Reference 100

Resolution: unresolved; no resolver link, observed 2026-08-05T12:35:51.957442Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

Observation fb443bfe-da23-4701-8a9a-d8ec61521e0f · inbound

Fantastic Pretraining Optimizers and Where to Find Them cites this paper.

Fantastic Pretraining Optimizers and Where to Find Them Benchmarking Optimizers for Large Language Model Pretraining

Reference 2025

Resolution: unresolved; no resolver link, observed 2026-08-05T11:59:49.778357Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 949e6136-8c02-4f11-92d0-9bccef329228 · inbound

Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods cites this paper.

Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods Benchmarking Optimizers for Large Language Model Pretraining

Reference 16

Resolution: verified exact; arxiv_id, observed 2026-05-21T21:10:38.756135Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation c30db491-0a24-427c-8803-312436a37102 · inbound

Optimal Sample Complexity for Single Time-Scale Actor-Critic with Momentum cites this paper.

Optimal Sample Complexity for Single Time-Scale Actor-Critic with Momentum Benchmarking Optimizers for Large Language Model Pretraining

Reference 50

Resolution: verified exact; arxiv_id, observed 2026-05-16T08:17:36.487221Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation cb893229-810e-4b4b-a8fb-5b78c166f127 · inbound

LoRDO: Distributed Low-Rank Optimization with Infrequent Communication cites this paper.

LoRDO: Distributed Low-Rank Optimization with Infrequent Communication Benchmarking Optimizers for Large Language Model Pretraining

Reference 2014

Resolution: unresolved; no resolver link, observed 2026-08-03T04:44:56.066179Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation bd78ae6a-878b-48cd-961f-806021dbee74 · inbound

HTMuon: Improving Muon via Heavy-Tailed Spectral Correction cites this paper.

HTMuon: Improving Muon via Heavy-Tailed Spectral Correction Benchmarking Optimizers for Large Language Model Pretraining

Reference 21

Resolution: verified exact; arxiv_id, observed 2026-05-25T06:55:26.265757Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation c9deeb18-9d70-42b9-8189-95b541cf1845 · inbound

Sharp Capacity Scaling of Spectral Optimizers in Learning Associative Memory cites this paper.

Sharp Capacity Scaling of Spectral Optimizers in Learning Associative Memory Benchmarking Optimizers for Large Language Model Pretraining

Reference 48

Resolution: verified exact; arxiv_id, observed 2026-05-14T23:38:16.407545Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation 2a8e356d-9410-4f35-93ef-67484520e0cb · inbound

Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima cites this paper.

Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima Benchmarking Optimizers for Large Language Model Pretraining

Reference 35

Resolution: verified exact; arxiv_id, observed 2026-05-11T05:55:57.177884Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation e4f7eebb-04aa-417e-872d-8d2b80194fb9 · inbound

Evolution of Optimization Methods: Algorithms, Scenarios, and Evaluations cites this paper.

Evolution of Optimization Methods: Algorithms, Scenarios, and Evaluations Benchmarking Optimizers for Large Language Model Pretraining

Reference 25

Resolution: verified exact; arxiv_id, observed 2026-05-11T10:51:20.489496Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation 32b87b17-45dd-41fe-b844-5e8b4f504424 · inbound

Benchmarking Optimizers for MLPs in Tabular Deep Learning cites this paper.

Benchmarking Optimizers for MLPs in Tabular Deep Learning Benchmarking Optimizers for Large Language Model Pretraining

Reference 11

Resolution: metadata mismatch; arxiv_id, observed 2026-05-10T12:10:22.021599Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation 47e8c762-ed4f-428b-87fc-c65a0ce4027a · inbound

StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models cites this paper.

StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models Benchmarking Optimizers for Large Language Model Pretraining

Reference 36

Resolution: verified exact; arxiv_id, observed 2026-05-10T12:15:22.218722Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation ab73c9d0-656a-4a0d-a2d1-9c563db4178b · inbound

When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$-norm Lower Bounds cites this paper.

When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$-norm Lower Bounds Benchmarking Optimizers for Large Language Model Pretraining

Reference 29

Resolution: verified exact; arxiv_id, observed 2026-05-11T19:21:07.675545Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation c241f457-713a-41f8-9d3e-7a495da4d34b · inbound

Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less cites this paper.

Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less Benchmarking Optimizers for Large Language Model Pretraining

Reference 23

Resolution: verified exact; arxiv_id, observed 2026-05-11T19:26:08.543311Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation 49fb2eb5-b6a4-4d35-a8c8-2845f077e2c9 · inbound

Navigating LLM Valley: From AdamW to Memory-Efficient and Matrix-Based Optimizers cites this paper.

Navigating LLM Valley: From AdamW to Memory-Efficient and Matrix-Based Optimizers Benchmarking Optimizers for Large Language Model Pretraining

Reference 37

Resolution: verified exact; arxiv_id, observed 2026-05-12T06:41:45.519353Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation 5883819f-1012-4d65-a32d-7cdeca1deba4 · inbound

Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity cites this paper.

Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity Benchmarking Optimizers for Large Language Model Pretraining

Reference 32

Resolution: verified exact; arxiv_id, observed 2026-05-14T19:32:50.855805Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation 68c55c15-3ee8-495a-88c3-3a8999dc5df4 · inbound

Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers cites this paper.

Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers Benchmarking Optimizers for Large Language Model Pretraining

Reference 134

Resolution: verified exact; arxiv_id, observed 2026-05-20T09:38:11.217214Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation 1750ef5d-27b2-4566-848b-b9b4cd74497a · inbound

Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers cites this paper.

Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers Benchmarking Optimizers for Large Language Model Pretraining

Reference 136

Resolution: verified exact; arxiv_id, observed 2026-06-30T18:45:00.393205Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation 56fd8440-a43e-42df-ad27-029d1533ea45 · inbound

LionMuon: Alternating Spectral and Sign Descent for Efficient Training cites this paper.

LionMuon: Alternating Spectral and Sign Descent for Efficient Training Benchmarking Optimizers for Large Language Model Pretraining

Reference 23

Resolution: verified exact; arxiv_id, observed 2026-05-20T07:28:06.765568Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation 8fd65943-b119-450d-a15c-bc72839058d5 · inbound

LionMuon: Alternating Spectral and Sign Descent for Efficient Training cites this paper.

LionMuon: Alternating Spectral and Sign Descent for Efficient Training Benchmarking Optimizers for Large Language Model Pretraining

Reference 23

Resolution: verified exact; arxiv_id, observed 2026-06-30T18:35:00.435900Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation cef63c76-0d48-4a8f-ab54-be36c37a7463 · inbound

Why SGD is not Brownian Motion: A New Perspective on Stochastic Dynamics cites this paper.

Why SGD is not Brownian Motion: A New Perspective on Stochastic Dynamics Benchmarking Optimizers for Large Language Model Pretraining

Reference 192

Resolution: verified exact; arxiv_id, observed 2026-05-22T08:11:17.313797Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation 36e17f08-5a44-462f-b932-356011a2b584 · inbound

Reparametrizing Shampoo and SOAP for Subspace Basis Updates and BFloat16 Storage cites this paper.

Reparametrizing Shampoo and SOAP for Subspace Basis Updates and BFloat16 Storage Benchmarking Optimizers for Large Language Model Pretraining

Reference 22

Resolution: verified exact; arxiv_id, observed 2026-06-29T22:24:00.706903Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation b9670b42-1ecb-444c-bd9d-520367ce8fa4 · inbound

Softsign: Smooth Sign in Your Optimizer For Better Parameter Heterogeneity Handling cites this paper.

Softsign: Smooth Sign in Your Optimizer For Better Parameter Heterogeneity Handling Benchmarking Optimizers for Large Language Model Pretraining

Reference 44

Resolution: verified exact; arxiv_id, observed 2026-06-28T22:52:44.860967Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation 670a7054-6c7e-4b71-b263-8d575561617c · inbound

Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering cites this paper.

Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering Benchmarking Optimizers for Large Language Model Pretraining

Reference 46

Resolution: verified exact; arxiv_id, observed 2026-07-02T01:56:27.815268Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation 68a78f89-2c6a-40fe-b4de-9242b71381ca · inbound

Open Problem: Is AdamW Effective Under Heavy-Tailed Noise? cites this paper.

Open Problem: Is AdamW Effective Under Heavy-Tailed Noise? Benchmarking Optimizers for Large Language Model Pretraining

Reference 7

Resolution: verified exact; arxiv_id, observed 2026-07-04T10:09:44.602263Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

Observation 9c5e31cf-cfa2-4bc6-ac80-97675cc31922 · inbound

OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers cites this paper.

OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers Benchmarking Optimizers for Large Language Model Pretraining

Reference 97

Resolution: unresolved; no resolver link, observed 2026-07-11T22:10:49.683444Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d3b04c17-064f-4229-9c1d-ccc1dcc9f23c · inbound

(A)iSpy: Parasitic Trojans for Machine Learning Infrastructure cites this paper.

(A)iSpy: Parasitic Trojans for Machine Learning Infrastructure Benchmarking Optimizers for Large Language Model Pretraining

Reference 71

Resolution: unresolved; no resolver link, observed 2026-08-01T17:47:03.790519Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d3922175-6ba6-404b-b07c-115aab2d1ab1 · inbound

PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer cites this paper.

PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer Benchmarking Optimizers for Large Language Model Pretraining

Reference 48

Resolution: unresolved; no resolver link, observed 2026-08-01T17:32:35.613381Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 970c7596-d80c-40ac-8060-6449687b52f1 · inbound

Scale Weight Decay and Train Better cites this paper.

Scale Weight Decay and Train Better Benchmarking Optimizers for Large Language Model Pretraining

Reference 67

Resolution: unresolved; no resolver link, observed 2026-07-30T12:53:41.174828Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 76c7a6db-6cbd-4f13-a74e-2226997896a9 · inbound

CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization cites this paper.

CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization Benchmarking Optimizers for Large Language Model Pretraining

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-04T06:04:04.927838Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.