Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay | Paper Citation Record

Observation 3c50d069-29ae-49bd-9ff6-db04e345740e · outbound

This paper cites L2 Regularization versus Batch and Weight Normalization.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay L2 Regularization versus Batch and Weight Normalization

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:40.509350Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 02af2d7f-3a1e-439c-8063-a303f00ffb55 · outbound

This paper cites International Conference on Learning Representations , year=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay International Conference on Learning Representations , year=

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:40.608270Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c694cf3e-1a96-4281-badc-4087eaca19d7 · outbound

This paper cites Norm matters: efficient and accurate normalization schemes in deep networks , volume =.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Norm matters: efficient and accurate normalization schemes in deep networks , volume =

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:40.772412Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8d485295-b39d-4449-8a27-c9c1623b37d7 · outbound

This paper cites Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate , volume =.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate , volume =

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:40.813367Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 24d3c7cc-ebaa-4c86-8b19-43d750c38325 · outbound

This paper cites Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction , volume =.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction , volume =

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:40.901249Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 08e0f05e-330e-4e87-b9c0-4f73ab606ea1 · outbound

This paper cites Online Normalization for Training Neural Networks , volume =.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Online Normalization for Training Neural Networks , volume =

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:40.969144Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 76092085-cc2a-4f20-912b-48f7d1c7f383 · outbound

This paper cites Spherical Motion Dynamics: Learning Dynamics of Normalized Neural Network using SGD and Weight Decay , volume =.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Spherical Motion Dynamics: Learning Dynamics of Normalized Neural Network using SGD and Weight Decay , volume =

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:41.037351Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 93e607d3-3560-4973-bbc3-58267e82bf76 · outbound

This paper cites An Exponential Learning Rate Schedule for Deep Learning.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay An Exponential Learning Rate Schedule for Deep Learning

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:41.107729Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 68b7475e-ba41-4cc1-bbea-e706ee9022d4 · outbound

This paper cites Proceedings of the AAAI Conference on Artificial Intelligence , author=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Proceedings of the AAAI Conference on Artificial Intelligence , author=

Reference 9

Resolution: verified exact; doi, observed 2026-08-01T08:53:55.686142Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 9683ace2-8bf8-4f70-a5a5-0d5a23522549 · outbound

This paper cites On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay , volume =.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay , volume =

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:41.215129Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 68263ae8-9407-4993-87c6-bf1d4b324fab · outbound

This paper cites Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes , volume =.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes , volume =

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:41.292037Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b4ac6085-7cef-4b35-9996-2631186ccb36 · outbound

This paper cites Greenwade.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Greenwade

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:41.381779Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4d66435b-848f-41be-9799-452086a5197b · outbound

This paper cites NeurIPS 2023 Workshop Heavy Tails in Machine Learning , year=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay NeurIPS 2023 Workshop Heavy Tails in Machine Learning , year=

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:41.456091Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ff1d4b52-464e-4c3c-a238-677888512589 · outbound

This paper cites The Break-Even Point on Optimization Trajectories of Deep Neural Networks.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay The Break-Even Point on Optimization Trajectories of Deep Neural Networks

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:41.552689Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation fdec7626-9c7a-41a8-ab6a-4a9f23ebe551 · outbound

This paper cites On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:41.633722Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 71e7f561-54f2-4a81-ad47-b6e249b99043 · outbound

This paper cites The large learning rate phase of deep learning: the catapult mechanism.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay The large learning rate phase of deep learning: the catapult mechanism

Reference 16

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:41.724748Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3714f7ea-bd17-4ed1-87e2-49b39a1e4753 · outbound

This paper cites On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:41.830960Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c89e712d-ff2f-49b4-88f0-0762b3ca8201 · outbound

This paper cites Transactions on Machine Learning Research , year=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Transactions on Machine Learning Research , year=

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:41.901561Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 78f4c3fa-ee27-4a87-847a-b237b81f01bf · outbound

This paper cites The 22nd international conference on artificial intelligence and statistics , pages=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay The 22nd international conference on artificial intelligence and statistics , pages=

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:41.982473Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 42128c5a-6378-4e36-bedd-45bf5ee9d41e · outbound

This paper cites International conference on artificial intelligence and statistics , pages=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay International conference on artificial intelligence and statistics , pages=

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:42.104439Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ce3c4d6e-f4b6-4b3b-b86d-f90355cdd05f · outbound

This paper cites A Simple Convergence Proof of Adam and Adagrad.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay A Simple Convergence Proof of Adam and Adagrad

Reference 21

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:42.201005Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 758981a4-69c2-4362-9dc1-abdc9518c5a4 · outbound

This paper cites Journal of Machine Learning Research , volume=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Journal of Machine Learning Research , volume=

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:42.263741Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c948cbfc-2532-45ac-8061-b77d7c3210d3 · outbound

This paper cites The Thirty-eighth Annual Conference on Neural Information Processing Systems , year =.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay The Thirty-eighth Annual Conference on Neural Information Processing Systems , year =

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:42.354268Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8c13421a-82ea-40c0-a15e-b246431cffde · outbound

This paper cites Journal of Machine Learning Research , volume=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Journal of Machine Learning Research , volume=

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:42.453785Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 07379f32-cdb1-41ca-a558-e6fe812b6eef · outbound

This paper cites Advances in neural information processing systems , volume=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Advances in neural information processing systems , volume=

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:42.613734Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 027e6191-2577-4b1c-b576-df742fce7f74 · outbound

This paper cites 2025 , eprint=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay 2025 , eprint=

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:42.697813Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b8afb3e1-896f-420e-9d3b-44e60f2f8444 · outbound

This paper cites Thirty-seventh Conference on Neural Information Processing Systems , year=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Thirty-seventh Conference on Neural Information Processing Systems , year=

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:42.745265Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation bdf12044-4cf5-4039-bed6-ae4388208f5f · outbound

This paper cites 2025 , institution =.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay 2025 , institution =

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:42.820521Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1de2a617-b0ad-4114-8c28-4022a0dcd908 · outbound

This paper cites International Conference on Learning Representations , year =.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay International Conference on Learning Representations , year =

Reference 29

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:42.872027Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f57efdc6-53bb-43a7-b87b-4f7727a429ac · outbound

This paper cites Advances in neural information processing systems , volume=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Advances in neural information processing systems , volume=

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:42.967292Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ace03bdf-55f9-4e0f-8f3b-68dc723eff14 · outbound

This paper cites 2025 , url=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay 2025 , url=

Reference 31

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:43.039395Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2c254a8e-d036-4ec5-ad04-b008189543f3 · outbound

This paper cites 2005 , edition =.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay 2005 , edition =

Reference 32

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:43.137199Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 291098ac-6303-4e3b-9c1c-f6473047bec8 · outbound

This paper cites Reasoning Bias of Next Token Prediction Training.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Reasoning Bias of Next Token Prediction Training

Reference 33

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:43.217179Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 37b17ac5-4854-409f-aee5-781a9abc04d6 · outbound

This paper cites Journal of Machine Learning , year =.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Journal of Machine Learning , year =

Reference 34

Resolution: verified exact; doi, observed 2026-08-01T08:53:55.457389Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 1196e2a3-5350-4e4f-9088-890ce11da6c1 · outbound

This paper cites International Conference on Learning Representations , year=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay International Conference on Learning Representations , year=

Reference 35

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:43.462766Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9911b3b7-a5b6-4ecb-af40-e75b4a2a9654 · outbound

This paper cites Advances in Neural Information Processing Systems , volume=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Advances in Neural Information Processing Systems , volume=

Reference 36

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:43.616177Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 47aea571-b516-4802-b46a-9e7e72b81ee9 · outbound

This paper cites Adam: A Method for Stochastic Optimization.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Adam: A Method for Stochastic Optimization

Reference 37

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:43.735821Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 55ba53fa-3430-4798-a09a-9830a5ea443f · outbound

This paper cites International conference on machine learning , pages=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay International conference on machine learning , pages=

Reference 38

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:43.887195Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c1c537cc-db42-4e59-b7a0-337ee42671e8 · outbound

This paper cites A Walk with SGD.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay A Walk with SGD

Reference 39

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:44.047334Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a4a02ef0-e0fc-4221-8d3b-93d8c90022ea · outbound

This paper cites Three Factors Influencing Minima in SGD.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Three Factors Influencing Minima in SGD

Reference 40

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:44.212716Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9fb2a540-1f8b-44c8-905f-a8b03782a904 · outbound

This paper cites an unresolved cited work.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Unresolved cited work

Reference 41

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:44.384006Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8d54af1b-5422-4e11-b159-07480aea771c · outbound

This paper cites Advances in Neural Information Processing Systems , volume=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Advances in Neural Information Processing Systems , volume=

Reference 42

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:44.507953Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8ea16c42-b9c2-4655-ab80-6e0add5bf808 · outbound

This paper cites The Eleventh International Conference on Learning Representations , year=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay The Eleventh International Conference on Learning Representations , year=

Reference 43

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:44.582789Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e51403e3-7f5a-45fa-86ae-2ab3682fa28e · outbound

This paper cites an unresolved cited work.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Unresolved cited work

Reference 44

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:44.721836Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c67a4cbd-1874-47ce-89b4-d4ce1edc6b65 · outbound

This paper cites International Conference on Learning Representations , year=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay International Conference on Learning Representations , year=

Reference 45

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:44.805354Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 074e021c-3e6e-45ed-aeed-19f11cff766c · outbound

This paper cites International Conference on Machine Learning , pages=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay International Conference on Machine Learning , pages=

Reference 46

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:44.889080Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6394b825-124d-4356-8885-492a8e51e54f · outbound

This paper cites Mathematical and scientific machine learning , pages=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Mathematical and scientific machine learning , pages=

Reference 47

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:45.004525Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 995b41c9-986f-4efa-99a4-12545112319d · outbound

This paper cites Journal of Computational Mathematics , year=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Journal of Computational Mathematics , year=

Reference 48

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:45.074313Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6eb4aa11-6139-429c-b41c-b22a835280b6 · outbound

This paper cites Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective

Reference 49

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:45.185948Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 38d7c8b4-5238-45c6-b7bb-fc753844a08f · outbound

This paper cites International Conference on Learning Representations , year=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay International Conference on Learning Representations , year=

Reference 50

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:45.256235Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d64b195a-336b-412b-92a0-02ec74bd23ae · outbound

This paper cites Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition , pages=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition , pages=

Reference 51

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:45.325431Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2279d5e6-0658-4bb4-80f7-9d0c7e81dbb5 · outbound

This paper cites A Theory on Adam Instability in Large-Scale Machine Learning.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay A Theory on Adam Instability in Large-Scale Machine Learning

Reference 52

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:45.390519Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b04247f0-7d18-47ad-8e38-b27b5e78fb73 · outbound

This paper cites Scale-Distribution Decoupling: Enabling Stable and Effective Training of Large Language Models.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Scale-Distribution Decoupling: Enabling Stable and Effective Training of Large Language Models

Reference 53

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:45.475754Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 05c4c720-1162-4aa0-ad63-e86d1da21025 · outbound

This paper cites Spike No More: Stabilizing the Pre-training of Large Language Models.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Spike No More: Stabilizing the Pre-training of Large Language Models

Reference 54

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:45.563563Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7003e75a-81ab-4830-83de-8026f736f2af · outbound

This paper cites International Conference on Machine Learning , pages=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay International Conference on Machine Learning , pages=

Reference 55

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:45.628065Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1bda72ad-9bb0-4c24-818d-bbf7513c6c5a · outbound

This paper cites On the Relation Between the Sharpest Directions of.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay On the Relation Between the Sharpest Directions of

Reference 56

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:45.735024Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 463879d1-1b8e-4207-a868-9cbb86002847 · outbound

This paper cites International Conference on Learning Representations , year=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay International Conference on Learning Representations , year=

Reference 57

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:45.856651Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 501dcaae-fa3c-4417-828b-c1ae8e3a766c · outbound

This paper cites Transactions on Machine Learning Research , issn=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Transactions on Machine Learning Research , issn=

Reference 58

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:45.938336Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a90d9349-bb77-413b-966f-94e9e16de469 · outbound

This paper cites Transactions on Machine Learning Research , issn=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Transactions on Machine Learning Research , issn=

Reference 59

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:46.070571Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0741cebe-0303-4481-84a5-c0990a7e405f · outbound

This paper cites 2021 , url=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay 2021 , url=

Reference 60

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:46.160491Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c52f06de-75ed-4eb4-9b2b-cc959c36fb97 · outbound

This paper cites Llama 2: Open Foundation and Fine-Tuned Chat Models.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Llama 2: Open Foundation and Fine-Tuned Chat Models

Reference 61

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:46.262324Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2c12c153-a591-4522-90f8-61a5fc4e4297 · outbound

This paper cites arXiv e-prints , pages=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay arXiv e-prints , pages=

Reference 62

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:46.335833Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ca577734-1e13-42bf-95e3-e31691acf32e · outbound

This paper cites arXiv preprint arXiv:2505.21829 , year=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay arXiv preprint arXiv:2505.21829 , year=

Reference 63

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:46.427622Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0142365c-fed8-4ea2-b5b8-5e5fe01b82ab · outbound

This paper cites The Thirteenth International Conference on Learning Representations , year=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay The Thirteenth International Conference on Learning Representations , year=

Reference 64

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:46.491540Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 957176aa-2270-489b-9f61-733038cc5650 · outbound

This paper cites Adaptive Preconditioners Trigger Loss Spikes in Adam.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Adaptive Preconditioners Trigger Loss Spikes in Adam

Reference 65

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:46.548954Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c47ef091-e82b-4c5f-aca7-ad541e404999 · outbound

This paper cites International conference on machine learning , pages=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay International conference on machine learning , pages=

Reference 66

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:46.664973Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d64acf75-c068-4165-855e-ed5f13e9b493 · outbound

This paper cites Layer Normalization.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Layer Normalization

Reference 67

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:46.759687Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1049a9d2-8ada-4c7c-80b5-7648383feee9 · outbound

This paper cites A very short proof of Cauchy's interlace theorem for eigenvalues of Hermitian matrices.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay A very short proof of Cauchy's interlace theorem for eigenvalues of Hermitian matrices

Reference 68

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:46.901311Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation dcb40e31-a58b-40ca-9309-ca5d2c705893 · outbound

This paper cites Proceedings of the IEEE conference on computer vision and pattern recognition , pages=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

Reference 69

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:46.981059Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3ee028f6-51cb-45ee-9faf-be8337f3ccc1 · outbound

This paper cites an unresolved cited work.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Unresolved cited work

Reference 70

Resolution: parse uncertain; no resolver link, observed 2026-08-01T08:50:47.051256Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 873ec28e-4416-4780-a8be-72c0de5024e1 · outbound

This paper cites Advances in neural information processing systems , volume=.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Advances in neural information processing systems , volume=

Reference 71

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:47.158492Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 481e2229-3afe-4d48-b379-5d8805499b69 · outbound

This paper cites ATT Labs [Online].

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay ATT Labs [Online]

Reference 72

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:47.249113Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cb575da1-722c-40a8-adc7-f92df1441ddb · outbound

This paper cites Decoupled Weight Decay Regularization.

Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay Decoupled Weight Decay Regularization

Reference 73

Resolution: unresolved; no resolver link, observed 2026-08-01T08:50:47.336816Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Coverage vector

Reference resolution

Outbound references

Pith citing papers