{"id":"7d8c26ae-b934-404c-b7c6-756998726203","arxiv_id":"2501.11584","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"GCSAM combines Gradient Centralization with Sharpness-Aware Minimization and reports modest, unrepeated test accuracy improvements over SAM and Adam in image classification.","lead":"The paper proposes GCSAM, a training method that applies Gradient Centralization inside Sharpness-Aware Minimization to stabilize gradients and speed up convergence. It reports test accuracy gains over SAM and Adam on CIFAR-10 and two medical imaging datasets, but the gains are small, single-run, and missing key experimental details.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Appendix A.1's purported PAC-Bayes bound for GCSAM is invalid: it replaces a maximum over the perturbation ball by the single centralized-gradient direction and infers tightness from a smaller pre-normalization gradient norm, although the normalized perturbation still has radius ρ.","rationale":"I read this as an empirical optimizer paper whose headline claim is consistent gains over SAM and Adam. For that claim to hold, either the empirical evidence must be strong, or the proposed mechanism must be credible. The empirical tables contain single numbers with no seeds, error bars, or explicit base-optimizer statement, so the evidence alone is not sufficient to carry the claim. The remaining support is Theorem A.1, which is supposed to show that GCSAM gives a tighter sharpness bound. That theorem fails at the step where the maximum over the full ℓ_p ball is replaced by the value at the centralized-gradient perturbation: the replacement has the wrong direction, and the norm inequality is irrelevant because the perturbation is renormalized to radius ρ. This is exactly the transfer the reader flagged, and it is load-bearing because it is the only stated reason why centralizing the ascent gradient should improve generalization. The concrete check is an independent re-derivation; if it fails, the theory collapses. The empirical weaknesses are also present, but this theoretical gap is the sharpest single point of failure. The reader's REJECT verdict remains appropriate, so no change is needed.","tokens_in":11319,"tokens_out":7185,"duration_ms":81700,"concrete_test":"Independently re-derive Theorem A.1 for the actual GCSAM objective (Eq. 6) from the Foret et al. PAC-Bayes theorem, keeping the maximum over the full ℓ_p ball and not substituting the single centralized-gradient direction. If the step replacing max_{||ϵ||_p≤ρ} L_S(w+ϵ) with L_S(w+ϵ_GC) cannot be justified, the claimed tighter sharpness bound does not follow; then the theoretical support for GCSAM's generalization improvement is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that GCSAM improves generalization rests on the assertion in §2.3 that centralizing the gradient yields tighter sharpness bounds, with Theorem A.1 given as the proof. The proof does not go through. After copying the Foret et al. PAC-Bayes inequality, it writes 'which can be bounded as' and replaces max_{||ϵ||_p≤ρ} L_S(w+ϵ) with L_S(w+ϵ_GC), where ϵ_GC is the single perturbation along the centralized gradient. This replacement is invalid: ϵ_GC is one point inside the perturbation ball, so the maximum over the full ball is greater than or equal to the value at that point, not less than or equal. The subsequent norm comparison is also a non-sequitur: while ||∇_GC L_S(w)||_2 ≤ ||∇L_S(w)||_2, the perturbation ϵ_GC is normalized to have norm ρ exactly, so the smaller pre-normalization gradient norm does not reduce the perturbation radius or the sharpness term. Thus the claimed tighter bound is unsupported, and the paper's only stated mechanism for GCSAM's generalization benefit collapses, leaving only single-run accuracy differences without error bars.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Gradient Centralized Sharpness-Aware Minimization (GCSAM), which applies Gradient Centralization (GC) to the gradient used in SAM's inner ascent step. It claims that centralizing the gradient reduces gradient noise and variance, leads to tighter sharpness bounds, and yields better test accuracy and computational efficiency than SAM and Adam on CIFAR-10, breast ultrasound, and COVID-19 chest X-ray datasets across ResNet50, VGG16, ViT, and Swin Transformer. A PAC-Bayes generalization bound is stated in Theorem A.1 as theoretical support.","tokens_in":11594,"tokens_out":4250,"duration_ms":45300,"significance":"If the central claims were sound, GCSAM would be a simple, potentially useful drop-in modification of SAM for image classification. The paper has strengths: it releases code, evaluates on both general and medical imaging benchmarks, compares against several SAM variants, and includes loss-landscape visualizations. However, the theoretical mechanism in Appendix A.1 is invalid, and the experimental evidence consists of single-run accuracy numbers without error bars, seeds, or a reported base optimizer for SAM/GCSAM. As a result, the paper's main claims are currently unsupported.","major_comments":[{"comment":"The proof of Theorem A.1 contains a direction error that invalidates the claimed bound. After copying the PAC-Bayes inequality from Foret et al., the manuscript replaces the expectation over Gaussian perturbations with max_{||ϵ||_p≤ρ} L_S(w+ϵ_GC), where ϵ_GC is a single perturbation along the centralized gradient. Since ϵ_GC is feasible for the original maximization, max_{||ϵ||_p≤ρ} L_S(w+ϵ) ≥ L_S(w+ϵ_GC), so this replacement gives a lower bound, not an upper bound. The subsequent norm comparison ||∇_GC L_S(w)||_2 ≤ ||∇L_S(w)||_2 does not reduce the perturbation radius because ϵ_GC is normalized to have norm exactly ρ. Thus the paper does not establish that GCSAM has a tighter sharpness bound than SAM, and the stated mechanism for GCSAM's generalization benefit is unsupported.","section":"Appendix A.1, Theorem A.1 proof"},{"comment":"All experimental comparisons are single runs without seeds, error bars, or statistical tests. Table 1 reports differences as small as 0.03 percentage points (ViT: 84.05 vs. 84.08), and Table 3 shows comparable gaps; without variance estimates or multiple seeds, the claim that GCSAM 'consistently outperforms' SAM and Adam is not established. The manuscript also never states which base optimizer (e.g., SGD or Adam) is used for SAM and GCSAM in the experiments, so it is impossible to verify that the comparison is fair. These omissions concern the paper's central empirical claim and should be addressed.","section":"§3, Tables 1–3"},{"comment":"The experimental protocol is under-specified: the learning-rate and ρ grid values, number of epochs, weight decay, data augmentation, and early-stopping criterion are not reported. Training 'until each model achieves 100% accuracy on the training set' is an unusual stopping rule that can interact with optimizer differences, and the paper does not say whether the same stopping rule and hyperparameters were used for every optimizer. This prevents reproducibility of the results in Tables 1–3.","section":"§3.1, experimental protocol"},{"comment":"The computational-efficiency claim is not supported by the reported speed numbers. Table 3 lists GCSAM speeds of 2.10, 1.07, 1.18, and 1.87 relative to Adam, while SAM speeds are 2.27, 1.28, 1.32, and 2.04; these are single measurements with no stated measurement methodology (hardware, batch size, number of steps, warm-up, or number of runs). In the ResNet50 and Swin Transformer rows, GCSAM is actually slower than SAM, which contradicts the blanket claim of improved efficiency.","section":"§4, 'Better Computational Efficiency'"}],"minor_comments":[{"comment":"The maximization in Eq. (6) is over ||ϵ_GC||_p ≤ ρ, but ϵ_GC is defined in Eq. (7) as a single normalized direction; it should be clarified whether the maximization is over all perturbations of that form or only the one computed at the current iterate.","section":"Eq. (6)"},{"comment":"The line 'Compute Descent Step w = w − α(gGC(w + ϵGC))' is ambiguous: it is not defined how gGC applies centralization to gradients evaluated at the perturbed weights, and the parentheses are unbalanced. This should be rewritten with clear notation.","section":"Algorithm 1"},{"comment":"The bound in Theorem A.1 introduces a parameter η that is not defined, uses ρ both as the perturbation radius and inside the Gaussian perturbation scaling, and the notation for the projection P in the theorem statement differs from the matrix P = I − ee^T used in the proof. These inconsistencies make the statement hard to parse.","section":"Theorem A.1"},{"comment":"The loss-landscape visualization is qualitative; the text claims GCSAM produces 'consistently flatter minima' but no quantitative sharpness measure (e.g., Hessian eigenvalue or the maximum loss in a ball of fixed radius) is reported.","section":"Fig. 3"},{"comment":"There are numerous typos and grammatical issues, including 'an important area' in the Introduction, 'paramter update' in the Fig. 2 caption, 'fot' in the same caption, and 'Efficieny' in the Section 4 heading. A careful copyedit is needed.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The central theoretical result is invalid, and the empirical evaluation lacks the statistical rigor needed to support the main claims. I do not see a path to acceptance without a new theoretical justification and a substantially more rigorous experimental protocol, both of which go beyond minor revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take on GCSAM. The algorithm is exactly what it says: centralize the gradient, use the centralized gradient to compute SAM's ascent perturbation, then update. Equations 6-7 and Algorithm 1 make this unambiguous. It is a legitimate composition, not a huge conceptual leap, but the paper is honest about that and the implementation is simple enough to reproduce. Credit where due: the medical imaging experiments cover BUS and COVIDx with four architectures, and they compare against several SAM variants as well as SAM and Adam. That is a reasonable empirical scope for a short optimizer paper.\n\nThe problem is the theory. The stress-test note holds up: Theorem A.1 does not prove what it claims. The proof copies Foret et al.'s PAC-Bayes inequality, then replaces max over the perturbation ball with L_S(w + epsilon_GC). Since epsilon_GC is one feasible point inside that ball, the max is greater than or equal to that value, so the inequality goes the wrong direction. The later norm comparison (||P grad|| <= ||grad||) does not rescue it, because epsilon_GC is normalized to have norm rho exactly. So the claimed tighter sharpness bound is unsupported. I looked for another argument in the text; there isn't one.\n\nEmpirically, there is a more basic issue: no seeds, no error bars, and no explicit statement of the base optimizer used for SAM and GCSAM in Tables 1-2. The pseudocode uses SGD, the prose says \"such as SGD or Adam,\" and the baseline is labeled Adam. If SAM and GCSAM were run with Adam, that should be stated. Without repeats or seeds, differences of 0.3-2.4 points are within plausible run-to-run noise. The speed comparisons in Table 3 are also single-run and hard to interpret.\n\nOverall, the algorithm is plausible and the experimental scope is broad, but the central theoretical claim collapses and the empirical support is too thin as it stands. A revision with the proof fixed or removed, proper seeds and error bars, and explicit base-optimizer settings could become a useful incremental contribution. I would not cite it in its current form, but the algorithmic idea is real enough that I would not desk-reject a revision.\n\nRecommendation: send to peer review, expecting major revision.","headline":"Neat but incremental SAM variant; the appendix's PAC-Bayes argument does not survive scrutiny, and the single-run empirical results are too thin to carry the central claim as written.","tokens_in":12082,"tokens_out":2040,"would_cite":false,"duration_ms":24203,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GCSAM claims that centralizing the ascent gradient in SAM produces flatter minima, better test accuracy, and lower computational cost than SAM and Adam.","keywords":["gradient centralization","sharpness-aware minimization","flat minima","generalization","image classification","medical imaging","PAC-Bayes bound","optimizer"],"falsifier":"Train a fixed architecture (e.g., ResNet-18) on CIFAR-10 with GCSAM, SAM, and Adam under identical schedules for at least 10 random seeds and compare test accuracy and time-to-target-accuracy; if GCSAM does not consistently beat or match SAM, the central claim is refuted. A targeted theoretical check would compute the PAC-Bayes bound of Theorem A.1 for a small two-layer network under both ε and ε_GC and verify whether the centralized bound actually holds.","tokens_in":11114,"feed_emoji":"🎯","tokens_out":5438,"duration_ms":53203,"temperature":0.7,"pith_summary":"This paper proposes GCSAM, an optimizer that inserts Gradient Centralization into the ascending step of Sharpness-Aware Minimization (SAM). The authors aim to establish that centering the ascent gradient before perturbing the weights both reduces sharpness more effectively and stabilizes training, so GCSAM attains higher test accuracy than SAM and Adam across general and medical image benchmarks while running faster than SAM. If true, GCSAM is a drop-in modification of an already widespread method, so the improvement would propagate to any SAM-based training pipeline. The supporting argument is a PAC-Bayes bound in which the centralized perturbation has smaller magnitude than the original gradient, plus empirical results on CIFAR-10, breast ultrasound, and COVID-19 chest X-ray datasets.","feed_headline":"Centered gradients give SAM a generalization edge","feed_subtitle":"GCSAM subtracts each gradient's mean before SAM's ascent, claiming better test accuracy at lower cost.","key_machinery":"The central object is the projection matrix P = I − ee^⊤ that subtracts the mean from each gradient vector, together with the perturbation rule ε_GC = ρ (P∇L_S(w))/||P∇L_S(w)||_p. The projection defines what the paper calls gradient centralization; applying it inside SAM's inner maximization yields a shorter ascent direction, which the paper uses to argue for a tighter sharpness bound (max over ε_GC of L_S(w+ε_GC) no larger than max over ε of L_S(w+ε)). The inequality ||P∇L_S(w)||^2 ≤ ||∇L_S(w)||^2 is the keystone of the proof and the algorithm.","core_discovery":"The central claim is that replacing the ascent gradient in SAM with its centralized version, ε_GC = ρ ∇_GC L_S(w)/||∇_GC L_S(w)||, does not sacrifice the sharpness-reduction guarantee and in fact tightens it, because ||∇_GC L_S(w)||^2 = ||∇L_S(w)||^2 − (e^⊤∇L_S(w))^2 ≤ ||∇L_S(w)||^2. From this inequality the paper derives a PAC-Bayes generalization bound for GCSAM and concludes that the method converges to flatter minima. Empirically, the paper reports that on four architectures GCSAM achieves higher test accuracy than Adam and SAM on CIFAR-10, breast ultrasound, and COVID-19 CXR-4 datasets, and that it trains faster than SAM on the medical benchmarks.","pith_inferences":["The proof's transition from max over ε to max over ε_GC inside the PAC-Bayes bound is the step most likely to fail; if it does, GCSAM's theoretical advantage reduces to an empirical regularity rather than a proven guarantee.","Because the mean subtraction depends on layer width, GCSAM's effect should be stronger for wide layers and nearly vanish for narrow bottleneck layers; this is a testable prediction about where the optimizer helps most.","The reported speedups relative to SAM may be partly due to the early-stopping rule at 100% training accuracy; a time-to-target-accuracy comparison without early stopping would separate convergence speed from the stopping protocol.","GCSAM's principle is orthogonal to other SAM variants such as ASAM and MSAM, so combining centralization with those modifications is a natural next test."],"forward_implications":["Any existing SAM-based pipeline can adopt GCSAM by changing only the ascent-gradient computation, so claimed gains would transfer across applications.","If the bound in Theorem A.1 holds, GCSAM inherits SAM's PAC-Bayes generalization guarantee with a sharper sharpness term.","The reduced gradient magnitude implies a built-in damping of gradient explosions during the ascent step, which may permit larger perturbation radii ρ.","Lower training time on vision transformers compared with SAM, if confirmed, would make sharpness-aware training more practical on large models.","GCSAM's accuracy gains over Adam, if replicated, offer a free accuracy improvement for image classifiers without extra hyperparameter tuning."],"supporting_citations":[{"why":"Supplies the sharpness-aware minimization objective and the PAC-Bayes generalization bound that GCSAM modifies.","marker":"[12]"},{"why":"Supplies the gradient centralization projection P = I − ee^⊤ that forms the core of the ascent-step modification.","marker":"[49]"},{"why":"Motivates the approach by showing sharpness-based measures correlate with generalization.","marker":"[18]"},{"why":"Baseline SAM variant whose performance on COVID-19 data GCSAM must beat for the comparison claim.","marker":"[28]"},{"why":"Baseline SAM variant compared in the COVID-19 experiments.","marker":"[52]"},{"why":"Baseline SAM variant compared in the COVID-19 experiments.","marker":"[46]"},{"why":"Baseline SAM variant compared in the COVID-19 experiments.","marker":"[4]"}],"fun_headline_variants":["Centered gradients sharpen SAM's edge","GCSAM: faster, flatter, better","Gradient centralization boosts SAM","SAM's ascent, now gradient-centered"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's theoretical tightening of the sharpness bound rests on assuming that the PAC-Bayes inequality from SAM remains valid when the perturbation is restricted to the centralized-gradient direction, a step the appendix passes over quickly.","fun_headline_variants_meta":{"raw":{"variants":["Centered gradients sharpen SAM's edge","GCSAM: faster, flatter, better","Gradient centralization boosts SAM","SAM's ascent, now gradient-centered"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000166,"raw_usage":{"total_tokens":1223,"prompt_tokens":887,"completion_tokens":336,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":503,"completion_tokens_details":{"reasoning_tokens":283}},"tokens_in":503,"tokens_out":336,"duration_ms":4696,"temperature":1.0,"reasoning_tokens":283,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:04:34.360826+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a fixed architecture (e.g., ResNet-18) on CIFAR-10 with GCSAM, SAM, and Adam under identical schedules for at least 10 random seeds and compare test accuracy and time-to-target-accuracy; if GCSAM does not consistently beat or match SAM, the central claim is refuted. A targeted theoretical check would compute the PAC-Bayes bound of Theorem A.1 for a small two-layer network under both ε and ε_GC and verify whether the centralized bound actually holds.","supporting_citations":[{"cited_title":"Sharpness-aware minimization for efficiently improving generalization","cited_arxiv_id":null,"evidence_quote":"Supplies the sharpness-aware minimization objective and the PAC-Bayes generalization bound that GCSAM modifies."},{"cited_title":"Gradient centralization: A new optimization tech- nique for deep neural networks","cited_arxiv_id":null,"evidence_quote":"Supplies the gradient centralization projection P = I − ee^⊤ that forms the core of the ascent-step modification."},{"cited_title":"Fantastic generalization mea- sures and where to find them","cited_arxiv_id":null,"evidence_quote":"Motivates the approach by showing sharpness-based measures correlate with generalization."},{"cited_title":"Asam: Adaptive sharpness-aware minimiza- tion for scale-invariant learning of deep neural networks","cited_arxiv_id":null,"evidence_quote":"Baseline SAM variant whose performance on COVID-19 data GCSAM must beat for the comparison claim."},{"cited_title":"Surrogate gap minimization im- proves sharpness-aware training","cited_arxiv_id":null,"evidence_quote":"Baseline SAM variant compared in the COVID-19 experiments."},{"cited_title":"Cr-sam: Curva- ture regularized sharpness-aware minimization","cited_arxiv_id":null,"evidence_quote":"Baseline SAM variant compared in the COVID-19 experiments."},{"cited_title":"Momentum-sam: Sharpness aware minimization without computational overhead","cited_arxiv_id":null,"evidence_quote":"Baseline SAM variant compared in the COVID-19 experiments."}],"review_version":1}