{"id":"9cb7aa22-fa1d-4f31-8057-9a61baa570a4","arxiv_id":"2505.06597","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Increasing L2 regularization produces additional first-order accuracy transitions beyond the known onset-of-learning, and the paper interprets grokking as hysteresis across these transitions.","lead":"This paper argues that increasing L2 regularization makes deep networks jump between accuracy regimes, visible as sudden changes in the curvature of the error surface. It then links the delayed learning phenomenon called grokking to hysteresis from these first-order transitions.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Grokking-as-hysteresis is asserted, not demonstrated: Fig. 3 shows initialization-dependent slow convergence at fixed β, and no β up-down sweep or natural grokking test is reported.","rationale":"The reader's weakest_assumption focuses on data-complexity scaling, and I agree that premise is weakly supported. However, an even more direct load-bearing gap is that the term 'hysteresis' is never actually tested: there is no β down-sweep, no coexistence loop, only fixed-β initialization dependence. This is the key step connecting the observed phase transitions to grokking. The concrete β up-down test would settle whether the delayed convergence in Fig. 3 is genuinely history-dependent in the sense of a first-order transition. The data-complexity premise remains relevant because the paper's proposed route to natural grokking depends on it, but the hysteresis demonstration is more fundamental to the 'nothing but' claim. The reader's CONDITIONAL verdict remains appropriate: the concern is substantial but addressable, and the basic observations of transitions may still be real. Re-running the experiments with a proper annealing loop and, ideally, a known natural-grokking task would either support or refute the central mechanism.","tokens_in":12098,"tokens_out":14956,"duration_ms":143950,"concrete_test":"Directly test hysteresis in the same Setup-1 architecture: anneal β upward from 0 through β0 and β1 while training, then continue from the final parameters and anneal β back downward to 0, recording the accuracy/error as a function of β in both directions. Repeat for at least 10 seeds and a pre-registered change-point detector. If the downward transition happens at a different β than the upward transition (a reproducible loop), hysteresis is demonstrated; if the two curves coincide, the Fig. 3 delay is just slow escape from a chosen initial condition and the grokking-as-hysteresis claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central explanatory claim—'grokking is nothing but a hysteresis effect associated with a first-order phase transition'—is not supported by the reported experiments. In the Grokking section and Fig. 3, β is fixed below β1 and only the initial condition is varied; the low- and intermediate-phase initializations take longer to escape. This is initial-condition dependence, not hysteresis: a hysteresis loop requires sweeping the control parameter β in both directions and showing that the transition point depends on the direction of the sweep. No such reverse sweep is presented (the annealing mode described in the experimental setup is only used to initialize successive forward runs). The paper itself concedes that natural grokking is 'unlikely to occur' in this setup and that the connection to random-initialization grokking at high data complexity is only hypothesized. The load-bearing premise behind that hypothesis—that increasing data complexity adds accuracy basins, so 'the number of transitions thus increases'—is supported only by a 1D-vs-2D Gaussian comparison; the MNIST result changes the loss to cross-entropy and the architecture/task, so it does not isolate data complexity. If either the hysteresis interpretation or the basin-count premise fails, the unified explanation of grokking and the prediction of new transition points collapse.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes that L2-regularized deep neural networks undergo first-order phase transitions between distinct accuracy regimes, that these transitions are detectable as change-points in the Ricci scalar curvature of the error surface, and that the number of such transitions increases with data complexity. The authors report numerical experiments on Gaussian regression tasks (1D and 2D outputs), MNIST classification, a one-hidden-layer variant, and a VAE-like model with KL regularization, and they offer an explanation of grokking as a hysteresis effect caused by a model being trapped in a lower-accuracy basin. The paper includes a first-principles derivation of a Ricci-curvature formula for error surfaces and shows qualitative agreement between jumps in the error, jumps in the distance from the origin, and discontinuities in the curvature.","tokens_in":12306,"tokens_out":11433,"duration_ms":99852,"significance":"The paper's main contribution would be a geometric, parameter-free diagnostic for accuracy phase transitions and a unified explanation of delayed convergence in deep networks. The explicit curvature formula, stated as derived from the Gauss equation, is a potentially useful contribution, and the observation that curvature change-points coincide with accuracy transitions is interesting. The claimed connection to grokking is ambitious and, if supported by proper hysteresis experiments, would be significant. However, the current evidence is largely single-run and correlational; the load-bearing hysteresis claim and the basin-count prediction are not yet established. The paper does not fit any free parameter to the observed transition locations, which is a strength, but the empirical base is too thin to support the breadth of the claims.","major_comments":[{"comment":"The central claim that 'grokking is nothing but a hysteresis effect' is not supported by the reported experiments. In Fig. 3 the regularization strength beta is fixed at a sub-critical value and only the initial condition is varied; the green and orange curves show slower escape from low-accuracy initializations. Hysteresis in a first-order transition requires sweeping the control parameter beta in both directions and demonstrating that the transition point depends on the sweep direction. Since no reverse sweep or annealing-down experiment is reported, the delayed convergence is initial-condition dependence, not a demonstrated hysteresis loop. The manuscript itself concedes that natural grokking is 'unlikely to occur' in this setup, so the connection to actual grokking remains hypothetical.","section":"Grokking section, Fig. 3"},{"comment":"The scalar-curvature formula appears to have incorrect denominators. Starting from the authors' own definitions g^{ij} = delta^{ij} - partial^i l partial^j l / ||nabla F||^2 and II_{ij} = H_{ij}/||nabla F||, the Gauss equation gives R = [tr(H)^2 - tr(H^2)]/||nabla F||^2 + 2/||nabla F||^4 nabla l^T (H^2 - tr(H)H) nabla l, not the printed expression with denominators ||nabla F|| and ||nabla F||^2. A direct expansion of the authors' Supplementary Eqs. (15) and (16) yields this corrected form. If the numerical experiments used the printed formula, the reported Ricci-scalar values in Figs. 2 and 4 and the derived change-points are affected; if the correct formula was used, the manuscript should be corrected to avoid propagating the error.","section":"Eq. (3) and Supplementary Eq. (17)"},{"comment":"The prediction that the number of transitions increases with data complexity is not isolated. The 1D-versus-2D Gaussian comparison changes the output dimension and hence the target distribution; the MNIST experiment changes the loss from MSE to cross-entropy, the input dimension, and the task, so any increase in transition count cannot be attributed to data complexity alone. The paper states this premise as hypothesis (iii) without a derivation or a controlled experiment, yet it is load-bearing for the prediction of new transition points. A minimal control, such as fixing the architecture and loss while varying only the complexity of the Gaussian target, would be needed.","section":"Supplementary 'Varying data-complexity' and MNIST section"},{"comment":"The main curves appear to be single runs without error bars, multiple random seeds, or a description of the change-point detection algorithm (the MNIST caption says only 'some change-point detection algorithm'). Because the central evidence is the existence and location of discontinuities in noisy-looking curves, the absence of uncertainty quantification and a reproducible detection procedure makes it difficult to judge whether the reported beta0 and beta1 values are robust. The authors should specify the detection algorithm, its thresholds, and the statistics over repeated runs.","section":"Results and Supplementary 'Experimental Setup'"}],"minor_comments":[{"comment":"The title contains a typo ('Neur al'), and references [4] and [21] are duplicates of the same Ziyin and Ueda paper; these should be cleaned up.","section":"Title and reference list"},{"comment":"The phrase 'β j β1' is incomplete; it should read 'β ≪ β1' or similar. The lowercase spelling of 'grokking' throughout is also inconsistent with standard usage.","section":"Grokking section, first paragraph"},{"comment":"The paper states that the full Hessian matrices are saved for each trained model but does not report how the Hessian is computed (e.g., exact, Hutchinson, or finite differences), which is important for reproducibility of the curvature values.","section":"Completeness of the experimental description"},{"comment":"The conclusion acknowledges that the global structure of the error landscape cannot be deduced from the reported experiments; this limitation should be reflected earlier in the paper when the basin-count hypothesis is introduced.","section":"Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reports an interesting preliminary exploration, but the breadth of the claims exceeds the evidence. In particular, the hysteresis interpretation of grokking and the data-complexity prediction need controlled experiments, and the curvature formula needs to be corrected or its implementation verified. If the authors can address these load-bearing points, the paper could become suitable for publication; I do not see grounds for rejecting the underlying observation of accuracy transitions and curvature change-points."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read for you: the Ricci-curvature diagnostic is genuine, but the grokking-hysteresis claim is asserted rather than demonstrated. The paper reproduces Ziyin and Ueda's onset-of-learning transition, adds a second transition in a 2D Gaussian regression task, and reports multiple transitions on MNIST. The curvature calculation is the real contribution: they derive the scalar Ricci curvature of the error surface from the Gauss equation, giving a formula in terms of gradients and Hessian. That derivation is transparent and reproducible, and the concurrence of curvature change-points with accuracy jumps is a legitimate empirical observation.\n\nWhere it strains is the grokking claim. Figure 3 shows three initializations at fixed beta below the transition; one escapes quickly, two take thousands of epochs. That is initial-condition dependence, not hysteresis. Hysteresis requires sweeping beta up and down and showing the transition point depends on sweep direction. There is no reverse sweep anywhere. The paper itself says grokking is unlikely to occur in this setup and that the connection to natural grokking is only hypothesized, yet the abstract says the results provide 'a natural explanation' and the text says grokking 'is nothing but' a hysteresis effect. That is overreach.\n\nAlso, the predicted increase in transition count with data complexity is one 1D-vs-2D Gaussian comparison; the MNIST case changes the loss to cross-entropy, so it does not isolate complexity. The change-point detection is literally 'some change-point detection algorithm' in the text, with no details, no thresholds, no error bars, no seeds, and no code. Basic reproducibility is not there yet.\n\nWho should read it: people working on phase transitions in training dynamics or grokking will find the curvature diagnostic useful as a probe, but should not take the hysteresis story on faith. It deserves a serious referee because the geometric observation is interesting enough, but the authors need to either run a real hysteresis sweep or drop the 'nothing but' language, and they need to report seeds and specify the change-point detector. I would not cite it in its current form.","headline":"Worth a read for the curvature diagnostic, but the grokking-hysteresis claim is overreach: the experiment shows initialization dependence at fixed beta, not a hysteresis loop.","tokens_in":12848,"tokens_out":2164,"would_cite":false,"duration_ms":22674,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","53B20","82B26"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that L2 regularization drives deep networks through a series of first-order accuracy transitions, each marked by a sudden change in scalar (Ricci) curvature, and that grokking is the hysteresis of such a transition.","keywords":["L2 regularization","phase transitions","deep neural networks","grokking","Ricci curvature","error landscape","hysteresis","onset of learning"],"falsifier":"Train the same architecture on a family of datasets whose complexity is controlled (for instance, Gaussian inputs with output dimension 1, 2, 4, 8) and sweep $\\beta$ for each; if the number of curvature change-points and accuracy jumps does not increase with output dimension, or if an accuracy jump occurs with no simultaneous curvature change-point, then the geometric explanation fails.","tokens_in":11849,"feed_emoji":"🧠","tokens_out":6253,"duration_ms":55995,"temperature":0.7,"pith_summary":"The paper sets out to show that the accuracy of an L2-regularized deep network does not degrade smoothly as regularization strength grows: it jumps between distinct accuracy phases, and those jumps are first-order phase transitions. The authors trace the mechanism to the local geometry of the error landscape, specifically to sudden changes in scalar (Ricci) curvature at the model's location in parameter space. They argue that this geometric picture explains the onset-of-learning, predicts additional transitions when data complexity rises, and accounts for grokking as hysteresis: the model trapped in a lower-accuracy basin. If the picture holds, curvature measurements offer a probe of what a network has learned without invoking high-level notions like memorization or compression.","feed_headline":"Sweeping L2 regularization exposes hidden accuracy phases","feed_subtitle":"Curvature change-points mark each jump in accuracy; grokking appears as hysteresis.","key_machinery":"The central object is the scalar (Ricci) curvature $R$ of the error surface, viewed as a $d$-dimensional submanifold embedded in the $(d+1)$-dimensional space of parameters plus error. For mean-squared error the paper derives $R = \\frac{1}{\\|\\nabla F\\|}(\\operatorname{tr}(H)^2 - \\operatorname{tr}(H^2)) + \\frac{2}{\\|\\nabla F\\|^2}\\nabla l^T (H^2 - \\operatorname{tr}(H)H)\\nabla l$, involving only the error gradient $\\nabla l$ and Hessian $H$. A change-point in $R$ as $\\beta$ varies is taken as the geometric signature of crossing an accuracy-basin boundary; its stability in practice is why the authors use the Ricci scalar instead of the Gauss-Kronecker curvature.","core_discovery":"The paper claims that increasing L2 regularization strength $\\beta$ pushes a trained network through a sequence of accuracy basins, each separated by a first-order phase transition. The jump occurs when the global minimum of the regularized loss shifts past the boundary of an error-landscape basin, forcing the model to a different region of parameter space; the transition is registered as a simultaneous jump in error, in distance from the origin, and in the scalar Ricci curvature computed at the model point. The same mechanism, the authors argue, explains grokking: a model initialized in a lower-accuracy basin can remain trapped there for many epochs because the geometry of the error surface presents a barrier, until it escapes—a hysteresis effect rather than a change in memorization or compression. They support this with experiments on Gaussian toy data and MNIST, and with a deliberately induced grokking-like delay in a small network.","pith_inferences":["Editorial extension: if curvature change-points mark basin boundaries, one could test basin nesting by initializing models at different $\\beta$-grown parameter positions and measuring escape times; the hysteresis picture predicts escape time should diverge near transition points.","The supplementary relation between the induced metric and the Fisher information metric means the curvature probe could connect accuracy phases to information-geometric accounts of learning, a link the paper does not develop.","Testable extension: on a dataset with tunable input or output dimension, count curvature change-points as a function of dimension; monotonic growth would support the paper's data-complexity claim, while saturation or non-monotonicity would bound it."],"forward_implications":["Sweeping $\\beta$ and recording curvature change-points gives a practical way to map accuracy phases, including phases beyond the onset-of-learning.","Higher data complexity should produce more accuracy basins and therefore more transitions; the paper observes this direction from one-dimensional Gaussian output to two-dimensional output and to MNIST classification.","Grokking should appear naturally in sufficiently complex tasks even with random initialization, because the probability of landing in a low-accuracy basin increases.","The same geometric account should apply to regularizers other than L2: the KL-regularized VAE-like model shows transitions with the same signature, since the error-landscape geometry is shared.","Single-hidden-layer networks should show continuous (second-order-like) transitions rather than jumps, matching the paper's observation."],"supporting_citations":[{"why":"Supplies the observed onset-of-learning first-order phase transition that the paper reproduces and extends.","marker":"[4]"},{"why":"Links grokking to a first-order phase transition in two-layer networks, the result the paper reinterprets in geometric terms.","marker":"[7]"},{"why":"Documents grokking beyond algorithmic data, motivating the claim that task complexity drives the phenomenon.","marker":"[9]"},{"why":"Provides an alternative derivation of the scalar curvature formula that the paper's stable Ricci expression builds on.","marker":"[24]"},{"why":"Characterizes feature learning in L2-regularized deep networks, supporting the setting in which the transitions are studied.","marker":"[6]"},{"why":"Identifies saddle points as dominant in high-dimensional non-convex loss landscapes, background for why basins can trap trajectories.","marker":"[12]"},{"why":"Establishes that multilayer loss surfaces contain many local minima of similar value, the structural premise for distinct accuracy basins.","marker":"[13]"},{"why":"Studies the complexity dynamics of grokking, supporting the link between task complexity and delayed convergence.","marker":"[8]"},{"why":"Documents glassy dynamics in deep networks, providing evidence that loss landscapes can trap training dynamics.","marker":"[5]"}],"fun_headline_variants":["L2 regularization drives first-order accuracy phase transitions","Grokking explained by hysteresis in L2-regularized nets","Ricci curvature marks phase jumps in deep networks","L2 regularization reveals hidden accuracy phases and hysteresis","Hysteresis explains grokking in L2-regularized networks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that the number of transitions grows with data complexity rests on the assumption that more complex data creates more distinct accuracy basins in the error landscape, which the paper infers from a small set of experiments rather than derives.","fun_headline_variants_meta":{"raw":{"variants":["L2 regularization drives first-order accuracy phase transitions","Grokking explained by hysteresis in L2-regularized nets","Ricci curvature marks phase jumps in deep networks","L2 regularization reveals hidden accuracy phases and hysteresis","Hysteresis explains grokking in L2-regularized networks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000756,"raw_usage":{"total_tokens":3310,"prompt_tokens":842,"completion_tokens":2468,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":458,"completion_tokens_details":{"reasoning_tokens":2402}},"tokens_in":458,"tokens_out":2468,"duration_ms":16413,"temperature":1.0,"reasoning_tokens":2402,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:38:09.537024+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same architecture on a family of datasets whose complexity is controlled (for instance, Gaussian inputs with output dimension 1, 2, 4, 8) and sweep $\\beta$ for each; if the number of curvature change-points and accuracy jumps does not increase with output dimension, or if an accuracy jump occurs with no simultaneous curvature change-point, then the geometric explanation fails.","supporting_citations":[{"cited_title":"Zeroth, first, and second-order phase transitions in deep neural networks,","cited_arxiv_id":null,"evidence_quote":"Supplies the observed onset-of-learning first-order phase transition that the paper reproduces and extends."},{"cited_title":"& Tegmark, M","cited_arxiv_id":null,"evidence_quote":"Documents grokking beyond algorithmic data, motivating the claim that task complexity drives the phenomenon."},{"cited_title":"Identifying and attacking the saddle point problem in high-dimensional non-convex optimization,","cited_arxiv_id":null,"evidence_quote":"Identifies saddle points as dominant in high-dimensional non-convex loss landscapes, background for why basins can trap trajectories."},{"cited_title":"The loss surfaces of multilayer net- works,","cited_arxiv_id":null,"evidence_quote":"Establishes that multilayer loss surfaces contain many local minima of similar value, the structural premise for distinct accuracy basins."},{"cited_title":"& Pos- ner, I","cited_arxiv_id":null,"evidence_quote":"Studies the complexity dynamics of grokking, supporting the link between task complexity and delayed convergence."},{"cited_title":"& Janssen, L","cited_arxiv_id":null,"evidence_quote":"Documents glassy dynamics in deep networks, providing evidence that loss landscapes can trap training dynamics."}],"review_version":1}