{"paper":{"title":"Why Adam Can Beat SGD: Second-Moment Normalization Yields Sharper Tails","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Adam achieves δ^{-1/2} high-probability convergence while SGD requires at least δ^{-1} under bounded variance.","cross_cats":["cs.AI"],"primary_cat":"cs.LG","authors_text":"Ruinan Jin, Shaofeng Zou, Yingbin Liang","submitted_at":"2026-03-03T15:34:51Z","abstract_excerpt":"Despite Adam demonstrating faster empirical convergence than SGD in many applications, much of the existing theory yields guarantees essentially comparable to those of SGD, leaving the empirical performance gap insufficiently explained. In this paper, we uncover a key second-moment normalization in Adam and develop a stopping-time/martingale analysis that provably distinguishes Adam from SGD under the classical bounded variance model (a second moment assumption). In particular, we establish the first theoretical separation between the high-probability convergence behaviors of the two methods: "},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"we establish the first theoretical separation between the high-probability convergence behaviors of the two methods: Adam achieves a δ^{-1/2} dependence on the confidence parameter δ, whereas corresponding high-probability guarantee for SGD necessarily incurs at least a δ^{-1} dependence.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"the classical bounded variance model (a second moment assumption)","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Adam attains a δ^{-1/2} high-probability rate while any SGD guarantee must incur at least δ^{-1} dependence.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Adam achieves δ^{-1/2} high-probability convergence while SGD requires at least δ^{-1} under bounded variance.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"1ff2ae02e009c0f81fca54eba62fb3203975e7e7503f7d21e277724c679b580e"},"source":{"id":"2603.03099","kind":"arxiv","version":8},"verdict":{"id":"ff16dbb2-0f66-4c5f-a41e-a93ec93f46d7","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-15T16:29:40.792470Z","strongest_claim":"we establish the first theoretical separation between the high-probability convergence behaviors of the two methods: Adam achieves a δ^{-1/2} dependence on the confidence parameter δ, whereas corresponding high-probability guarantee for SGD necessarily incurs at least a δ^{-1} dependence.","one_line_summary":"Adam attains a δ^{-1/2} high-probability rate while any SGD guarantee must incur at least δ^{-1} dependence.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"the classical bounded variance model (a second moment assumption)","pith_extraction_headline":"Adam achieves δ^{-1/2} high-probability convergence while SGD requires at least δ^{-1} under bounded variance."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2603.03099/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}