{"paper":{"title":"A Fourier perspective on the learning dynamics of neural networks: from sample complexities to mechanistic insights","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"Online SGD cannot learn phase-only classification on isotropic high-dimensional inputs before order N cubed steps, but power-law spectra accelerate it substantially.","cross_cats":["cond-mat.dis-nn","cond-mat.stat-mech","cs.LG","math.PR"],"primary_cat":"stat.ML","authors_text":"Claudia Merger, Fabiola Ricci, Sebastian Goldt","submitted_at":"2026-05-16T10:03:41Z","abstract_excerpt":"Neural networks trained with gradient-based methods exhibit a strong simplicity bias: they learn simpler statistical features of their data before moving to more complex features. Previous analyses of this phenomenon have largely focused on settings with (quasi-)isotropic inputs. In this work, we study the simplicity bias from a Fourier perspective, which allows us to include two key features of natural images in the analysis: approximate translation-invariance and power-law spectra. We first show experimentally that simple neural networks trained on image classification tasks first rely on am"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"For isotropic and high-dimensional inputs, classification based on phase information alone is a genuinely hard task: online SGD cannot distinguish the structured inputs from noise within n ≪ N³ steps, but needs at least n ≫ N³ log²N steps.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The introduced synthetic data model for translation-invariant inputs faithfully captures the interaction between amplitudes, phases, and learning dynamics without introducing artifacts that invalidate the hardness result or the power-law acceleration claim.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Neural networks prioritize amplitude over phase in Fourier space during training on translation-invariant data; power-law spectra accelerate phase learning despite not aiding classification.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Online SGD cannot learn phase-only classification on isotropic high-dimensional inputs before order N cubed steps, but power-law spectra accelerate it substantially.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"6611edc52f38979eec3582f9a7b0ab57961b5a561a4cdd732d35d2bb35bad8ad"},"source":{"id":"2605.16913","kind":"arxiv","version":1},"verdict":{"id":"b07deb0d-611c-4111-bf7f-005316dcd5e4","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-19T19:36:43.567585Z","strongest_claim":"For isotropic and high-dimensional inputs, classification based on phase information alone is a genuinely hard task: online SGD cannot distinguish the structured inputs from noise within n ≪ N³ steps, but needs at least n ≫ N³ log²N steps.","one_line_summary":"Neural networks prioritize amplitude over phase in Fourier space during training on translation-invariant data; power-law spectra accelerate phase learning despite not aiding classification.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The introduced synthetic data model for translation-invariant inputs faithfully captures the interaction between amplitudes, phases, and learning dynamics without introducing artifacts that invalidate the hardness result or the power-law acceleration claim.","pith_extraction_headline":"Online SGD cannot learn phase-only classification on isotropic high-dimensional inputs before order N cubed steps, but power-law spectra accelerate it substantially."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2605.16913/integrity.json","findings":[],"available":true,"detectors_run":[{"name":"cited_work_retraction","ran_at":"2026-05-19T20:52:02.966853Z","status":"completed","version":"1.0.0","findings_count":0},{"name":"doi_title_agreement","ran_at":"2026-05-19T20:01:18.966218Z","status":"completed","version":"1.0.0","findings_count":0},{"name":"doi_compliance","ran_at":"2026-05-19T19:50:42.466159Z","status":"completed","version":"1.0.0","findings_count":0},{"name":"claim_evidence","ran_at":"2026-05-19T18:41:56.267720Z","status":"completed","version":"1.0.0","findings_count":0},{"name":"ai_meta_artifact","ran_at":"2026-05-19T18:33:26.347687Z","status":"skipped","version":"1.0.0","findings_count":0}],"snapshot_sha256":"bad1206fe8f822f674b123f4c5e618faf1a708c6a46cb649529c6270280635ff"},"references":{"count":69,"sample":[{"doi":"","year":2019,"title":"et al.SGD on Neural Networks Learns Functions of Increasing ComplexityinAdvances in Neural Information Processing Systems32(2019), 3491–3501","work_id":"53683ea2-598d-4167-a3f8-cd1ee846db79","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2022,"title":"Ingrosso, A. & Goldt, S. Data-driven emergence of convolutional structure in neural networks. Proceedings of the National Academy of Sciences119(2022)","work_id":"17bb55d4-69d2-4baa-ae29-e65e92b2d05f","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2023,"title":"& Goldt, S.Neural networks trained with SGD learn distributions of increasing complexityinInternational Conference on Machine Learning(2023), 28843–28863","work_id":"ef98ae7f-c1c1-4dc6-a96c-003eb64b3d6c","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2024,"title":"& Goldt, S.A distributional simplicity bias in the learning dynamics of transformersinAdvances in Neural Information Processing Systems37(2024), 96207–96228","work_id":"f1b76ea0-2af5-45de-9547-0e463c9ccdea","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2024,"title":"& Fern, X.Neural Networks Learn Statistics of Increasing Complexityin (arXiv, 2024)","work_id":"3d70ac82-d6a4-4a01-b1ac-fa87c1aa4d57","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":69,"snapshot_sha256":"84229301d22c2d48eda220452f8ebec61a76b86afad4ad887306e0c207f93bf3","internal_anchors":0},"formal_canon":{"evidence_count":2,"snapshot_sha256":"15b45c0048917a93b5b8d40ca6a64ac866edba30c2d610c80219203f27e1ad76"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}