{"id":"23026c8c-a4ae-4d07-8085-0469353376f7","arxiv_id":"2608.03197","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"SAM's largest Hessian eigenvalue is bounded by the cube root of bGamma/(2*rho*eta^2), so larger radius, smaller batch, or larger learning rate restrict linearly stable minima to flatter regions.","lead":"A theory paper derives a quantitative bound on how flat the minima found by Sharpness-Aware Minimization (SAM) can be, as a function of batch size, learning rate, and perturbation radius. It also introduces an adaptive SAM variant that tunes the radius by monitoring Taylor-approximation error and reports modest gains on CIFAR benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1 is the paper's central result, but its proof is missing from the manuscript; the main text stops at the SAM update approximation and Lemma 1 without deriving the cubic bound, so the core claim is unverified.","rationale":"The paper's central claim is a quantitative sharpness bound for SAM. For this claim to hold, Theorem 1 must be proved from Lemma 1 and the assumptions. The manuscript does not provide this proof; the text goes from the update approximation and the covariance lemma straight to the theorem statement, with all derivations deferred to an absent supplement. This is a load-bearing omission because the bound is the foundation for the empirical validation and the proposed TLC-SAM. The reader's identified weak assumption—the 'nearly decoupled' approximation behind Lemma 1—is important: if that model of gradient-noise covariance is wrong, the SAM-specific terms change and the bound fails. But even granting Lemma 1, the missing derivation is the immediate blocker. The empirical results only test monotonic associations, not the quantitative inequality, so they cannot rescue the absent proof. We agree with the REJECT verdict; a revised version with a self-contained proof, or at least a verifiable supplement, could be conditionally accepted.","tokens_in":15027,"tokens_out":16434,"duration_ms":196948,"concrete_test":"Obtain the supplementary material and verify the proof of Theorem 1; if it is unavailable, independently re-derive (10) from Lemma 1 and Definition 1 for the linearized quadratic model. Specifically, substitute the covariance expression (9) into the linear-stability condition E[L(w_{t+1})|w_t] ≤ L(w_t) and solve for the largest allowed eigenvalue; check whether the resulting inequality is λ_max^3 ≤ b E||∇L(w_t)||/(2 ρ η^2). If the derivation requires an assumption not listed in Assumptions 1–3 (e.g., a bound on L(w_t)/||∇L(w_t)||), or yields a different exponent or coefficient, then Theorem 1 as stated is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Locate: Theoretical Evidence, Theorem 1. The theorem is stated with 'we have' immediately after Lemma 1, but no derivation is given. The only concrete steps are Eq. (7)-(8), which approximate the SAM update and define the noise ξ_t, and Lemma 1, which states an approximate covariance. There is no step connecting Definition 1's linear-stability condition or Lemma 1's covariance to inequality (10); the proof is explicitly deferred to a supplementary material not included in this manuscript. This matters because Theorem 1's bound is the entire theoretical contribution: it motivates the empirical correlation study, the TLC-SAM design, and the qualitative conclusions. The bound's cube-root dependence on ρ and η^2 and its use of E||∇L(w_t)|| are non-obvious; checking it is not a formality, since Lemma 1 itself relies on an unverified 'nearly decoupled' approximation. Without the proof, the central claim is unsupported, and correlation evidence cannot substitute for a derivation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a linear-stability analysis of mini-batch Sharpness-Aware Minimization (SAM) near interpolating global minima. Its central claim is Theorem 1: every linearly stable minimum satisfies λ_max ≤ (b E[||∇L(w_t)||]/(2ρη^2))^{1/3} ≤ (bΓ/(2ρη^2))^{1/3}, where b is batch size, η is learning rate, ρ is the perturbation radius, and Γ bounds the gradient norm. The paper also reports a large empirical study (900 models on CIFAR-100 with ResNet-18 and VGG-19) claiming to validate the predicted negative association between ρ and the largest Hessian eigenvalue, and introduces TLC-SAM, a radius-adaptation method based on Taylor-locality error.","tokens_in":15203,"tokens_out":4185,"duration_ms":50750,"significance":"If Theorem 1 were proved and its assumptions verified, the result would provide a quantitative, testable link between SAM's hyperparameters and the curvature of its stable minima, potentially guiding hyperparameter selection and the design of SAM variants. The paper also contains a substantial controlled empirical study and a practical adaptive method, TLC-SAM, which are strengths. However, the central theorem is stated without a proof, the key noise-covariance lemma rests on an informal and unverified 'nearly decoupled' assumption, and the empirical evidence is mostly correlational rather than a direct test of the bound. The current contribution is therefore a promising conjecture plus supporting trends, not an established quantitative theory.","major_comments":[{"comment":"The central result is stated without proof. After Lemma 1, the text simply says 'we have' and displays the cubic bound, with no derivation connecting Definition 1's linear-stability condition or Lemma 1's covariance to inequality (10). The proof is deferred to a supplementary material that is not included. Since the cubic dependence on ρ, η, and b and the use of E||∇L(w_t)|| are non-obvious, this missing proof makes the main theoretical claim unsupported as presented.","section":"Theoretical Evidence, Theorem 1 (Eq. 10)"},{"comment":"Assumption 1 states E[||∇L(w_t)||^2] ≤ Γ, but Theorem 1's second inequality uses E[||∇L(w_t)||] ≤ Γ. By Jensen, E||∇L|| ≤ sqrt(E||∇L||^2) ≤ sqrt(Γ). The displayed upper bound should therefore contain sqrt(Γ), not Γ, unless the assumption is restated as a bound on the first moment. This is a load-bearing inconsistency in the main inequality.","section":"Assumption 1 and Theorem 1 (Eq. 10)"},{"comment":"Lemma 1's covariance formula (9) is derived under an informal 'nearly decoupled' condition on ∇f(x_i; w_t) and L_i(w_t), which is not stated among Assumptions 1–3 and is not verified for the CIFAR-scale networks used in the experiments. Assumption 2 says the full-batch gradient is negligible compared with per-sample gradients, which near an interpolating minimum is plausible only if per-sample gradients remain nonzero; the exact noise model is crucial because the ρ-dependent terms in (9) are what make Theorem 1 SAM-specific. Without a precise and justified noise model, the theorem's derivation is incomplete.","section":"Lemma 1 and Assumptions 2–3"},{"comment":"The empirical validation reports a negative Spearman correlation between ρ and λ_max. However, Theorem 1 is an upper bound on λ_max under linear stability assumptions, not a guarantee that increasing ρ strictly decreases λ_max for arbitrary trained models. The bound also depends on E||∇L(w_t)|| (or Γ), which is not measured or controlled. The reported correlations are consistent with the theorem but do not test its quantitative form, so they cannot substitute for the missing derivation.","section":"Empirical Evidence, Table 1"},{"comment":"Table 2 correlates test accuracy with (b/(2ρη^2))^{1/3} after pooling over all hyperparameter settings. But Theorem 1 makes no direct prediction about test accuracy; it predicts an upper bound on λ_max for linearly stable minima, and Γ is not fixed across runs. This pooled correlation is therefore only a loose heuristic check and does not validate the bound.","section":"Empirical Evidence, Table 2"}],"minor_comments":[{"comment":"Notation: H(w_t) is symmetric, so H(w_t)^⊤ is redundant; also the expression mixes matrices and norms without dimensional clarity. Please clarify the exact meaning of the terms.","section":"Eq. (9)"},{"comment":"The expectation E[·] is not explicitly over the mini-batch randomness; please specify the probability space.","section":"Assumption 1"},{"comment":"The figure caption says 'Independent x/y axes' and many panels appear to lack numerical axis labels; this makes the visual trend hard to assess. Please provide full axis labels or larger panels.","section":"Figure 1"},{"comment":"The table mixes results from different papers with different training epochs (e.g., 300-epoch PyramidNet entries). The footnote cautions against strict ranking, but the table layout invites direct comparison; consider separating the authors' own runs from literature values more clearly.","section":"Table 3"},{"comment":"Some references are incomplete, e.g., 'Wu, L.; Ma, C.; et al. 2018' should list all authors. Also, the abstract and body contain typesetting artifacts such as '3 p' before formulas; please correct them.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The central theorem is currently unproved in the submitted manuscript, and there is a concrete inconsistency between Assumption 1 and the Γ in Eq. (10). These are fixable in principle, which is why I recommend major revision rather than rejection. However, the authors must supply the full proof, correct the Γ step (or restate the assumption), and substantially reframe the empirical section as a consistency check rather than a validation of the quantitative bound."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper has a genuinely interesting idea — extending the SGD noise-alignment stability analysis of Wu, Wang, and Su (2022) to mini-batch SAM and getting the clean bound λ_max ≤ (bΓ/(2ρη²))^{1/3}. If the derivation holds, that's a useful quantitative statement about how ρ, batch size, and learning rate jointly constrain flatness, and it makes a real point about the stability-locality trade-off for ρ. The direction is right.\n\nWhat is genuinely good: the setup is careful. Assumptions are explicit (interpolating minima, small gradient norm, local linearization), the linear-stability definition is clear, and the authors are candid about the limited scope — they state up front that the analysis is local and near-interpolating. The controlled study of 900 models across two architectures and a grid of batch sizes and learning rates is real work; Table 1 mostly shows a consistent negative association between ρ and the largest Hessian eigenvalue. TLC-SAM, adjusting ρ based on Taylor-locality error, is a reasonable instantiation of the trade-off and the reported results are plausible.\n\nThe soft spots are not minor. The central theorem appears as a statement without proof. The main text goes from Lemma 1's covariance formula straight to 'we have' inequality (10), and the derivation is explicitly deferred to a supplement that isn't included. For a theory paper, that is a load-bearing gap: I can't check whether the cube-root dependence on ρ and η² actually follows from the noise model. Lemma 1 itself rests on a 'nearly decoupled' approximation between per-sample gradients and losses, which is an unverified modeling assumption for real trained networks. So the core claim is currently unverified.\n\nThe empirical section also does not test the bound. A decreasing upper bound does not imply that the observed λ_max itself decreases monotonically with ρ; that's a correlation with the direction of the inequality, not the inequality's content. The pooled accuracy correlation with ∛(b/(2ρη²)) drops the Γ term and conflates learning-rate variation with the theoretical factor. The experiments show a consistent trend, which is encouraging, but they don't validate the specific formula. The paper also does not quantitatively position its bound against the existing SAM bounds in Bartlett et al. (2023) or Long and Bartlett (2024), so the exact novelty margin is unclear.\n\nNet: the authors clearly know the literature and the direction is promising, but as presented this is a claim-without-derivation plus supporting-but-not-decisive experiments. It deserves a serious referee — I'd send it out rather than desk-reject — but I would not accept it without a self-contained proof, a check of Lemma 1's assumptions, and an experimental design that isolates the bound's predictions (error bars on Hessian estimates, released code and data). I wouldn't cite it yet, but I'd follow it if the supplement appears.\n\nBest,\n[Your name]","headline":"Theorem 1 looks like a promising extension of SGD linear stability to SAM, but the proof is deferred to a missing supplement and the empirical test doesn't isolate the bound, so the paper is not yet ready.","tokens_in":15742,"tokens_out":3678,"would_cite":false,"duration_ms":37935,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Mini-batch SAM provably favors flat minima: the sharpest stable curvature is capped by a cube-root bound in batch size, learning rate, and perturbation radius.","keywords":["sharpness-aware minimization","implicit bias","flat minima","linear stability","Hessian eigenvalue","hyperparameter bounds","generalization","SAM perturbation radius"],"falsifier":"Train a genuinely over-parameterized network (for example ResNet-18 on CIFAR-100) with fixed batch size and learning rate, measure $\\Gamma$ and the largest Hessian eigenvalue at the final minimum over a sweep of $\\rho$, and check whether $\\lambda_{\\max}$ ever exceeds $\\sqrt[3]{b\\Gamma/(2\\rho\\eta^2)}$ while the Taylor-locality error stays small. A single violation within the stable local regime refutes the theorem. Alternatively, estimate the SAM gradient-noise covariance empirically and test whether the $\\rho$-dependent cross term is present at the predicted magnitude.","tokens_in":14861,"feed_emoji":"📉","tokens_out":9003,"duration_ms":101112,"temperature":0.7,"pith_summary":"This paper sets out to explain quantitatively why Sharpness-Aware Minimization (SAM) lands in flat minima, and how the perturbation radius $\\rho$ combines with batch size and learning rate rather than acting alone. It proves that a linearly stable interpolating minimum of mini-batch SAM must have largest Hessian eigenvalue $\\lambda_{\\max}$ no larger than $\\sqrt[3]{b\\Gamma/(2\\rho\\eta^2)}$, where $b$ is batch size, $\\eta$ is learning rate, and $\\Gamma$ bounds the gradient norm. The bound says the flatness bias is governed by the ratio $b/(\\rho\\eta^2)$: smaller batches, larger learning rates, and larger (but still local) radii all restrict SAM to flatter minima. A controlled study of 900 CIFAR-100 models finds the predicted negative association between $\\rho$ and the largest Hessian eigenvalue, and a theory-derived variant, TLC-SAM, adapts $\\rho$ by monitoring Taylor-approximation error and further lowers the top eigenvalue.","feed_headline":"SAM's flatness bias obeys a cube-root bound","feed_subtitle":"A new bound links Hessian sharpness to batch size, learning rate, and SAM radius; 900 models support it.","key_machinery":"The key object is Lemma 1, an approximate formula for the covariance of SAM's mini-batch gradient noise near an interpolating minimum, $\\Sigma(w_t)\\approx 2L(w_t)\\big(G(w_t)+2\\rho\\,G(w_t)H(w_t)^\\top/\\|\\nabla L(w_t)\\|+\\rho^2 H(w_t)G(w_t)H(w_t)/\\|\\nabla L(w_t)\\|^2\\big)$. Inserting this covariance into the linear-stability condition for the linearized squared loss produces the cubic bound; the SAM perturbation $\\hat{\\varepsilon}=\\rho\\nabla L/\\|\\nabla L\\|$ enters only through the $\\rho$ and $\\rho^2$ coefficients, which is why the bound has $\\rho$ in the denominator.","core_discovery":"The central claim is a theorem: for mini-batch SAM near an interpolating global minimum, under local linearization, squared loss, and a gradient-noise alignment assumption, every linearly stable solution obeys $\\lambda_{\\max} \\le \\sqrt[3]{b\\,\\mathbb{E}[\\|\\nabla L(w_t)\\|]/(2\\rho\\eta^2)} \\le \\sqrt[3]{b\\Gamma/(2\\rho\\eta^2)}$. The middle term is the sharpened form, using the expected gradient norm; the right-hand side is the clean bound using the uniform gradient bound $\\Gamma$. The paper reads this as a quantitative mechanism: SAM's adversarial perturbation injects $\\rho$-dependent terms into the stochastic-gradient noise covariance, and linear stability then forbids directions whose curvature","pith_inferences":["The paper reports a pooled rank correlation with $\\sqrt[3]{b/(2\\rho\\eta^2)}$, but does not test whether contours of measured $\\lambda_{\\max}$ collapse onto that product; a direct grid experiment varying $b,\\eta,\\rho$ and plotting $\\lambda_{\\max}$ against $b/(\\rho\\eta^2)$ would test whether the bound is tight, not just order-preserving.","If the $\\rho$-dependent noise covariance is the actual mechanism, full-batch SAM (where the mini-batch noise term vanishes) should show much weaker flatness bias; this ablation is not in the paper.","The Taylor-locality error $E_t$ could serve as a per-point confidence measure for trusting the local bound, and the same controller could be applied to other SAM variants beyond radius adaptation.","The bound's scaling suggests $\\rho$ should grow roughly as $\\eta^{-2}$ to hold flatness fixed across a learning-rate schedule; a schedule that grows $\\rho$ late in training could preserve flatness as $\\eta$ decays, a direction the theory points to but does not study."],"forward_implications":["Flatness of SAM solutions is set by the ratio $b/(\\rho\\eta^2)$: users can trade batch size and learning rate against perturbation radius while keeping the same sharpness cap.","Increasing $\\rho$ flattens the minimum only while the first-order Taylor model remains valid; beyond that, linear stability and safe training are lost.","SAM variants such as ASAM and F-SAM can be seen as ways to enlarge the effective radius or to reduce descent interference, thereby tightening the same bound.","A radius controller that expands $\\rho$ when the Taylor approximation is accurate can lower the largest Hessian eigenvalue relative to fixed-radius SAM, as TLC-SAM demonstrates.","The largest Hessian eigenvalue is not the whole spectrum: SAM and TLC-SAM also redistribute spectral mass, lowering the dominant curvature while keeping a more dispersed spectrum than pure sharpness targeting would suggest."],"supporting_citations":[{"why":"Defines SAM's perturbed objective and gradient update; this is the algorithm whose stable minima the theorem constrains.","marker":"(Foret et al. 2021)"},{"why":"Provides the linear-stability condition and SGD gradient-noise covariance analysis that the paper extends to SAM's perturbation-dependent noise.","marker":"(Wu, Wang, and Su 2022)"},{"why":"Supplies the near-decoupling of per-sample loss and gradient used to derive Lemma 1's covariance formula.","marker":"(Mori et al. 2021)"},{"why":"Originates the dynamical-stability perspective that linearly stable SGD minima must be flat, the lens adopted here.","marker":"(Wu, Ma et al. 2018)"},{"why":"Establishes the local-linearization and input-smoothness framework invoked in Assumption 3.","marker":"(Ma and Ying 2021)"},{"why":"Hutchinson's method is used to estimate the largest Hessian eigenvalue in the CIFAR-100 experiments.","marker":"(Yao et al. 2020)"},{"why":"ASAM variant used to illustrate how a scale-aware relative radius yields a larger effective perturbation under the same locality budget.","marker":"(Kwon et al. 2021)"},{"why":"F-SAM variant used to illustrate how filtering the shared descent component increases perturbation variation and can make a larger radius locally usable.","marker":"(Li et al. 2024)"}],"fun_headline_variants":["Cube-root law governs SAM's flatness bias","SAM's sharpness capped by cube-root bound","Theorem ties SAM's flatness to batch size and rho","900 models validate SAM's cube-root flatness bound","New bound links SAM sharpness to hyperparameters"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that near an interpolating minimum the per-sample losses and gradients are nearly decoupled, so the SAM noise covariance really takes the algebraic form in Lemma 1; if that decoupling fails, the $\\rho$-dependent terms change and the cubic bound does not follow.","fun_headline_variants_meta":{"raw":{"variants":["Cube-root law governs SAM's flatness bias","SAM's sharpness capped by cube-root bound","Theorem ties SAM's flatness to batch size and rho","900 models validate SAM's cube-root flatness bound","New bound links SAM sharpness to hyperparameters"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00021,"raw_usage":{"total_tokens":1314,"prompt_tokens":880,"completion_tokens":434,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":624,"completion_tokens_details":{"reasoning_tokens":359}},"tokens_in":624,"tokens_out":434,"duration_ms":5321,"temperature":1.0,"reasoning_tokens":359,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T23:43:17.871338+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a genuinely over-parameterized network (for example ResNet-18 on CIFAR-100) with fixed batch size and learning rate, measure $\\Gamma$ and the largest Hessian eigenvalue at the final minimum over a sweep of $\\rho$, and check whether $\\lambda_{\\max}$ ever exceeds $\\sqrt[3]{b\\Gamma/(2\\rho\\eta^2)}$ while the Taylor-locality error stays small. A single violation within the stable local regime refutes the theorem. Alternatively, estimate the SAM gradient-noise covariance empirically and test whether the $\\rho$-dependent cross term is present at the predicted magnitude.","supporting_citations":[],"review_version":1}