{"id":"6b2654ed-71fc-4bf8-8280-cfe0816b13bd","arxiv_id":"2608.09523","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"The paper introduces H(psi)-convexity and H(Psi)-smoothness via Legendre functions, proves generalized GD/SGD convergence rates, and reformulates DNN training as composite optimization controlled by gradient energy and Jacobian induced norms.","lead":"This paper defines generalized smoothness and convexity using Legendre energy functions and claims a unified convergence theory for deep network training, including an optimal step size of exactly 1 for its generalized gradient descent. A generalist reader should care because the paper tries to explain why SGD succeeds on nonconvex neural network objectives without classical assumptions, and to derive architecture guidance from Jacobian singular values.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposition 3.3 is false for unbounded-parameter DNN objectives: the Taylor remainder depends on absolute parameter scale, so no translation-invariant energy functions can bound it; all downstream convergence claims inherit this failure.","rationale":"The reader's weakest_assumption—that global Legendre energies Ψ,ψ exist for every parameter pair of a DNN objective—is exactly the load-bearing condition. The paper supplies no construction, and the heuristic in Section 3.4 cannot work: radial energy functions depend only on the difference μ−ν, while first-order remainders of non-quadratic objectives also depend on absolute location. The w^4 example shows the claim is false outright on unbounded domains, which are the standard setting for DNN training. I agree with the reader's REJECT verdict: the central framework is not established. I give credit for plausible elements like the Jacobian-based bounds and the empirical correlation plots, but they do not rescue Proposition 3.3. A concrete falsifying test is easy and decisive, so the concern is not merely a matter of missing proof; the assertion is internally inconsistent with the definitions in the unbounded regime.","tokens_in":70045,"tokens_out":4423,"duration_ms":43475,"concrete_test":"Check the scalar objective G(w)=w^4 on R. Analytically compute S_G(μ,ν)=G(μ)-G(ν)-G'(ν)(μ-ν) = (μ-ν)(μ^3+μ^2ν+μν^2-3ν^3). For fixed δ=μ-ν, evaluate S_G(ν+δ,ν) at ν=0,10,100,1000 with δ=1; observe S_G ~ 6ν^2, confirming no bound by any function of δ alone. Then test a minimal network, e.g., loss L(w1,w2)=(w1 w2 x - y)^2 with fixed x,y, and show S along the w1 direction at fixed w2=ν grows like ν^2(w1-w1')^2, again violating Definition 3.2 on the unbounded parameter space R^2. If both checks confirm the failure, Proposition 3.3 and the downstream convergence theorems are invalid without additional boundedness or structural assumptions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim, Proposition 3.3, asserts that every DNN training objective is both H(Ψ)-smooth and H(ψ)-convex on parameter space. Definition 3.2 requires the first-order Taylor remainder S_G(μ,ν) to be bounded above and below by a radial energy function of μ−ν only. Section 3.4 claims this can always be achieved by increasing a_Ψ, r_Ψ, and c_Ψ. This is false on any unbounded domain: remainders of non-quadratic functions grow with absolute location, not just with the difference. Example: G(w)=w^4 has S_G(μ,ν)=(μ−ν)(μ^3+μ^2ν+μν^2−3ν^3). For fixed δ=μ−ν and ν→∞, S_G ~ 6ν^2δ, which is unbounded; no function Ψ(δ) can upper-bound it, and no convex ψ(δ) can lower-bound it when negative. The same pathology occurs in ReLU networks: scaling one weight in a multiplicative interaction changes the output quadratically, making the remainder depend on the scale of other parameters. Thus Proposition 3.3 is not merely unproven; it fails in the exact regime the paper targets. Theorems 4.1, 4.2, and 5.1 all inherit their applicability from this proposition, so the convergence guarantees and the claimed optimal learning rate of exactly 1 do not apply to general DNN objectives. The Jacobian-based bounds in Corollaries 5.2 and 5.3 are conditional on the same false premise, although the final inequality forms may be salvageable under explicit bounded-domain or quadratic-growth assumptions.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a generalized notion of convexity and smoothness based on Legendre energy functions: H(Ψ)-smoothness and H(ψ)-convexity (Definition 3.2). It then defines generalized GD and SGD through conjugate directions (Definitions 4.1 and 4.2), claims that generalized GD has an optimal learning rate of exactly 1 (Theorem 4.1), and derives convergence rates expressed in terms of gradient energy. A composite optimization formulation is used to argue that DNN training is governed by two quantities: gradient energy and the induced norm of the network Jacobian (Theorem 5.1, Corollaries 5.2 and 5.3). The empirical section reports correlations between training risk and the components of the proposed bounds across many architectures, datasets, optimizers, and losses. The central enabling result is Proposition 3.3, which asserts without proof that every DNN training objective is both H(Ψ)-smooth and H(ψ)-convex in the parameters.","tokens_in":70431,"tokens_out":6227,"duration_ms":59002,"significance":"If Proposition 3.3 were valid, the framework would indeed be broad: it would unify convex and nonconvex, smooth and nonsmooth objectives, and the duality in Proposition 3.2 is a mathematically natural extension of classical strong convexity/Lipschitz smoothness. The experimental campaign is extensive, covering multiple architectures, datasets, optimizers, and loss functions, and the appendix supplies detailed proofs for many auxiliary statements. However, the paper's central premise is asserted rather than proved, and it fails for non-quadratic objectives on unbounded parameter spaces. Since Theorems 4.1, 4.2, and 5.1 all inherit their applicability from Proposition 3.3, the convergence and optimal-learning-rate claims do not apply to general DNN objectives as stated. The claimed optimal learning rate of 1 is also a normalization built into the definition of the update direction rather than a fitted or predicted constant.","major_comments":[{"comment":"Proposition 3.3 is stated without proof and is not true for non-quadratic objectives on unbounded domains. Definition 3.2 requires the Taylor remainder S_G(µ,ν) to be bounded above and below by functions of the difference µ−ν only. For G(w)=w^4, S_G(µ,ν)=(µ−ν)(µ^3+µ^2ν+µν^2−3ν^3); fixing δ=µ−ν and letting ν grow shows S_G grows like 6ν^2δ, which is unbounded. No finite radial energy function Ψ(δ) can upper-bound this remainder, and no convex radial ψ(δ) can lower-bound it when the remainder is negative and unbounded below. The heuristic in Section 3.4 of increasing a_Ψ, r_Ψ, and c_Ψ cannot repair this. ReLU network objectives contain similar multiplicative interactions, so the proposition fails in exactly the regime the paper targets, and Theorems 4.1, 4.2, and 5.1 do not apply to general DNN objectives as stated.","section":"Section 3.2 (Proposition 3.3) and Section 3.4"},{"comment":"The claim that the optimal learning rate of generalized GD is exactly 1 is a normalization choice rather than a discovered property. The update direction in Eq. (14) is ∇Ψ*(∇G(θ_k)); when Ψ(·)=L/2‖·‖^2, this direction is ∇G(θ_k)/L, so α=1 corresponds to the classical step size 1/L. Corollary 4.1 confirms this explicitly. Thus the statement that the optimal learning rate is identically 1 is built into the definition of the conjugate direction, and it does not eliminate learning-rate selection for classical GD as the abstract suggests.","section":"Definition 4.1 and Theorem 4.1"},{"comment":"The convergence rates are expressed in terms of Ψ and ψ, which are chosen after the fact to majorize or minorize the Taylor remainder. Because any valid Ψ can be scaled (Proposition 3.2), the rates O(1/Ψ*(ε)) are not intrinsic properties of the objective; Section 3.4 concedes that computing a tight Ψ for a DNN is NP-hard. The rates therefore do not yield quantitative a priori predictions for DNN training. The empirical section only reports local Pearson correlations between the risk and the bound components, not the actual rate constants or the strict inequalities claimed in Corollary 5.3.","section":"Theorems 4.1/4.2, Sections 3.4 and 4.2"},{"comment":"The gradient correlation factor M defined in Eq. (25) is a maximum over the actual optimization trajectory, and the paper acknowledges it is generally intractable. The iteration bounds in Theorem 4.2 are expressed in terms of M, so they cannot be evaluated or empirically verified without an additional a priori bound on M. This is a load-bearing gap in the SGD convergence claims, not merely a presentation issue.","section":"Theorem 4.2 and Definition 4.3"}],"minor_comments":[{"comment":"The rendered figures contain unicode substitution artifacts such as \"/uni00000013/uni00000018\" in axis labels and legends, making several plots unreadable; they need to be regenerated.","section":"Figures 2–18"},{"comment":"The experiments validate the framework by measuring local Pearson correlations between log risk and bound components rather than checking whether the strict inequalities in Corollary 5.3 hold; the text should state this limitation explicitly.","section":"Section 6.3.2"},{"comment":"The upper bounds involve ‖J_θh_θ(z)^+‖ and divisions by σ_min(J_θf_θ(z)); these are finite only under a full-column-rank condition on the Jacobian, which is not stated in Definition 5.3. When σ_min=0 the bounds are vacuous, so the conditions under which the bounds are non-trivial should be made explicit.","section":"Theorem 5.1 / Corollary 5.3"},{"comment":"Setting the surrogate gradient to zero at non-differentiable points makes the statement 'it ensures that the assumption of zero gradient at the global optimum is satisfied' a convention, not a property of the objective; this should be clarified because a nonsmooth global minimizer may have nonzero subgradients.","section":"Section 2.2"}],"recommendation":"reject","confidential_remarks":"The rejection is driven by Proposition 3.3, which is the load-bearing claim for the whole paper and is both unproved and doubtful for unbounded non-quadratic objectives. The algebra of the auxiliary propositions appears internally consistent, and the experimental campaign is broad, but the experiments do not compensate for the absence of a valid premise. The generalized smoothness/convexity definitions and the duality result could form a useful framework if the paper were reframed under explicit bounded-domain or quadratic-growth assumptions, but as written the central claims about general DNN objectives are not supportable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: the framework is coherent but its load-bearing premise fails on unbounded parameter spaces. Proposition 3.3 asserts every DNN training objective is both H(Ψ)-smooth and H(ψ)-convex, where Definition 3.2 requires the Taylor remainder S_G(μ,ν) to be bounded by a radial energy function of μ−ν only. For nonquadratic objectives, the remainder depends on absolute parameter scale, not just the difference. G(w)=w^4 is a clean counterexample: for fixed δ=μ−ν, S_G grows like 6ν²δ as ν→∞, so no translation-invariant Ψ can upper-bound it and no convex ψ can lower-bound it when negative. ReLU networks show the same pathology through multiplicative interactions. The Section 3.4 heuristic—increase a, r, c—cannot fix this, because energy functions are translation-invariant. So the convergence theorems and the headline \"optimal learning rate = 1\" do not apply to general DNN objectives.\n\nWhat is genuinely useful: the duality between H(ψ)-convexity and H(Ψ)-smoothness (Proposition 3.2) and the gradient-energy/suboptimality bounds (Theorem 3.1) are cleanly derived. The composite bounds in Corollaries 5.2 and 5.3, relating risk to gradient energy and Jacobian singular values, are plausible and could be valuable under explicit bounded-domain or quadratic-growth assumptions. The experiments are extensive and honestly reported, but they validate correlations, not the framework's assumptions. The toy experiments on ∥θ∥^r confirm a normalization, not a discovery: the conjugate update defines α=1 into the algorithm.\n\nOther soft spots, in proportion: the paper itself concedes that computing the globally optimal Ψ for a DNN is NP-hard, yet it still assumes such Ψ exist—that is exactly the unproven step. The convergence rates depend on Ψ and ψ chosen after the fact to majorize/minorize remainders, with no construction for any practical network. No code or data is provided, though the derivations are mostly checkable.\n\nWho this is for: researchers working on generalized smoothness conditions or Jacobian-based risk bounds may find useful pieces, but the central claim is false for the regime the paper targets. As a referee I would recommend rejection with a clear invitation to resubmit a version that restricts the framework to bounded domains or polynomial-growth objectives and provides explicit energy constructions for at least one nontrivial network.","headline":"A coherent framework undone by a false central premise: no translation-invariant Legendre energies can bound Taylor remainders of DNN objectives on unbounded parameter spaces, so the convergence theorems do not apply as stated.","tokens_in":70951,"tokens_out":3064,"would_cite":false,"duration_ms":29845,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C26","90C25","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that every deep-network training objective is both generalized-convex and generalized-smooth, making the optimal learning rate of its generalized gradient descent exactly 1.","keywords":["generalized convexity","generalized smoothness","Legendre functions","conjugate duality","gradient energy","DNN training theory","SGD convergence analysis","Jacobian singular values"],"falsifier":"A concrete way to settle the claim: take a small network, such as a two-parameter ReLU network with squared loss on a fixed dataset, and test the two inequalities $S_G(\\mu,\\nu)\\leq \\Psi(\\mu-\\nu)$ and $S_G(\\mu,\\nu)\\geq \\psi(\\mu-\\nu)$ on a dense grid of parameter pairs spanning unbounded directions. Because $\\psi$ and $\\Psi$ are fixed convex radial functions, a single pair where the remainder violates either inequality for every admissible energy candidate refutes Proposition 3.3 for that objective, and with it the universality that Theorems 4.1, 4.2, and 5.1 assume. A cheaper probe is to move along a direction of negative curvature and measure whether $S_G(\\mu,\\nu)$ grows negatively without bound; if it does while $\\psi(0)$ must stay near zero, no convex radial lower bound can hold, and the framework's applicability to that network collapses.","tokens_in":69783,"feed_emoji":"🧮","tokens_out":15630,"duration_ms":119194,"temperature":0.7,"pith_summary":"The paper's project is to explain why stochastic gradient descent reliably trains deep neural networks whose objectives are neither convex nor smooth in the classical sense. It replaces the quadratic energy that underlies strong convexity and Lipschitz smoothness with a broad family of Legendre energy functions, defining $H(\\psi)$-convexity and $H(\\Psi)$-smoothness through upper and lower bounds on the first-order Taylor remainder $S_G(\\mu,\\nu)$. The central claim is that every DNN training objective is both $H(\\Psi)$-smooth and $H(\\psi)$-convex, that generalized GD then has optimal learning rate exactly 1, and that the empirical risk is governed jointly by the gradient energy $\\Psi^*(\\nabla G(\\theta))$ and the extreme singular values of the network Jacobian. If these claims hold, the framework unifies convex and non-convex, smooth and non-smooth objectives in one duality, and it would offer a principled account of why plain SGD finds low-risk solutions on real networks.","feed_headline":"Generalized gradient descent has optimal learning rate exactly 1","feed_subtitle":"Casting every DNN loss as both generalized-convex and generalized-smooth, one duality explains why SGD works.","key_machinery":"The central object is the energy function: a Legendre function $\\Psi$ with radial representation $\\Psi(\\mu)=\\Psi_\\circ(\\|\\mu\\|)$ and its unique minimum at the origin, whose convex conjugate $\\Psi^*$ is again an energy function. The generalized properties are defined by sandwiching the first-order Taylor remainder $S_G(\\mu,\\nu)=G(\\mu)-G(\\nu)-\\langle\\nabla G(\\nu),\\mu-\\nu\\rangle$ between $\\psi(\\mu-\\nu)$ and $\\Psi(\\mu-\\nu)$; the constant terms $\\psi(0)$ and $\\Psi(0)$ carry the degree of non-convexity and non-smoothness. The argument's engine is conjugate duality: the generalized GD update $\\theta_{k+1}:=\\theta_k-\\alpha(\\nabla G(\\theta_k))^*_{\\Psi^*}$ uses $\\nabla\\Psi^*$ to map the gradient to a dual direction, and the gradient energy $\\Psi^*(\\nabla G(\\theta))$ measures progress. The Fenchel–Young identity turns the dual update into a descent inequality that makes $\\alpha=1$ optimal, while the chain rule turns the composite objective's bounds into statements about the Jacobian's upper and lower induced norms.","core_discovery":"On the paper's own terms, the discovery is that classical strong convexity and Lipschitz smoothness are two halves of one duality: both bound how fast a function can vary through the quadratic term $\\|\\cdot\\|_2^2$, and convex conjugation converts one half into the other. Replacing that quadratic with a Legendre energy function $\\Psi$ (radial, minimized at the origin) yields generalized properties: $G$ is $H(\\Psi)$-smooth when $S_G(\\mu,\\nu) \\leq \\Psi(\\mu-\\nu)$ and $H(\\psi)$-convex when $S_G(\\mu,\\nu) \\geq \\psi(\\mu-\\nu)$. The paper proves the duality that $G$ is $H(\\Psi)$-smooth if and only if its convex conjugate $G^*$ is $H(\\Psi^*)$-convex, and proves Theorem 3.1, which traps the suboptimality gap $G(\\mu)-G^*$ between $\\Psi^*(\\nabla G(\\mu))$ and $\\psi^*(\\nabla G(\\mu))$. Generalized GD and SGD move along the conjugate-dual direction of the gradient, which yields a monotone descent inequality and forces the optimal learning rate to be $\\alpha=1$. For the composite structure of DNN training, the paper shows that convergence reduces to jointly decreasing the gradient energy and controlling the induced norm (extreme singular values) of the Jacobian, with explicit risk bounds for MSE, Softmax CrossEntropy, and Fenchel–Young losses, and architecture-level consequences for width, initialization, and skip connections.","pith_inferences":["The framework's usefulness will hinge on how tightly $\\psi(0)$ and $\\Psi(0)$ can be pinned: overly loose choices make the bounds vacuous, and the paper's own conjecture that finding the globally optimal $\\Psi$ is NP-hard suggests that usable constants for real networks will come from architecture-specific construction, not from global computation.","Because classical GD is generalized GD only when the update direction is the gradient, the theory suggests interpreting adaptive optimizers such as Adam as empirical approximations of the conjugate-dual direction; if so, learning-rate schedules in practice could be understood as surrogate dual steps.","A natural testable extension is to connect the Jacobian spectral bounds to flat-minima geometry: minima with large $\\sigma_{\\min}(J)$ would automatically have tight risk bounds, giving a spectral characterization of generalization-favoring flatness that the paper does not pursue."],"forward_implications":["Generalized GD with step size $\\alpha=1$ is monotonically descending in the sense $G(\\theta_{k+1})\\leq G(\\theta_k)-\\Psi^*(\\nabla G(\\theta_k))$, reaching gradient energy below $\\Psi^*(\\varepsilon)$ in $O(1/\\Psi^*(\\varepsilon))$ iterations.","With quadratic energies the framework recovers classical GD and SGD: optimal learning rate $1/L$, the classical condition number $L/\\sigma$, and the standard $O(\\kappa\\log(1/\\varepsilon))$ strongly-convex rates.","For SGD, the achievable gradient-energy floor is set by $(n-m)M/m$ and the suboptimality by the model capacity risk $R_G(s,m)$, predicting that batch size trades convergence speed against attainable solution quality, consistent with batch-size scheduling in practice.","For ERM with MSE or Softmax CrossEntropy, the empirical risk is sandwiched between gradient energy scaled by the squared extreme singular values of the Jacobian, e.g. $\\|\\nabla_\\theta L\\|_2^2/(2\\sigma_{\\max}^2)\\leq L(\\theta,z)\\leq \\|\\nabla_\\theta L\\|_2^2/(2\\sigma_{\\min}^2)$.","Architectural design is reinterpreted as spectral control: over-parameterization and random initialization shrink $\\sigma_{\\max}/\\sigma_{\\min}$ of the Jacobian, while skip connections prevent its extreme singular values from decaying with depth, keeping the risk bounds tight in deep networks."],"supporting_citations":[{"why":"Supplies the Legendre-function and convex-conjugate-duality facts (Proposition 8.1 and Lemma 8.4) on which the energy-function machinery and the smoothness–convexity duality are built.","marker":"Rockafellar, 1970"},{"why":"Supplies the classical strong-convexity/Lipschitz-smoothness duality (Lemma 8.2) that the paper generalizes and later recovers as a special case.","marker":"Kakade et al., 2009"},{"why":"Co-source for Lemma 8.2, the classical duality bounds that ground the motivating analogy for generalized convexity and smoothness.","marker":"Zhou, 2018"},{"why":"Provides the Fenchel–Young loss properties (Lemma 8.6) used to show that MSE, Softmax CrossEntropy, and other standard losses fit the generalized convexity and smoothness assumptions.","marker":"Blondel et al., 2020"},{"why":"Supplies the random-ball norm concentration bound (Lemma 8.7) that drives Proposition 5.1 on random initialization controlling the Jacobian's extreme singular values.","marker":"Blum et al., 2020"},{"why":"Provides the KL-divergence upper bound that fixes the concrete H(Psi)-smooth and H(psi)-convex constants for Softmax CrossEntropy in Corollary 5.3.","marker":"Csiszar and Talata, 2006"}],"fun_headline_variants":["Optimal learning rate is 1 for any DNN loss, thanks to duality","One duality unifies convex and non-convex DNN training","Generalized GD hits optimal step size exactly 1","Why SGD works: one duality for all DNN losses","Duality cracks DNN optimization, fixing step size at 1"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is Proposition 3.3: for every neural-network training objective there exist global convex Legendre energies $\\Psi$ and $\\psi$ such that the Taylor remainder obeys $\\psi(\\mu-\\nu)\\leq S_G(\\mu,\\nu)\\leq \\Psi(\\mu-\\nu)$ for every pair of parameter values, with $\\psi(0)$ and $\\Psi(0)$ close enough to zero to keep the bounds useful. The paper asserts this without construction, and the heuristic in Section 3.4 of increasing the constants $a$, $r$, and $c$ cannot certify a convex lower bound on a non-convex remainder over an unbounded parameter space; every convergence theorem inherits its applicability from this assumption.","fun_headline_variants_meta":{"raw":{"variants":["Optimal learning rate is 1 for any DNN loss, thanks to duality","One duality unifies convex and non-convex DNN training","Generalized GD hits optimal step size exactly 1","Why SGD works: one duality for all DNN losses","Duality cracks DNN optimization, fixing step size at 1"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000521,"raw_usage":{"total_tokens":2630,"prompt_tokens":1164,"completion_tokens":1466,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":780,"completion_tokens_details":{"reasoning_tokens":1378}},"tokens_in":780,"tokens_out":1466,"duration_ms":8736,"temperature":1.0,"reasoning_tokens":1378,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:42:59.171509+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete way to settle the claim: take a small network, such as a two-parameter ReLU network with squared loss on a fixed dataset, and test the two inequalities $S_G(\\mu,\\nu)\\leq \\Psi(\\mu-\\nu)$ and $S_G(\\mu,\\nu)\\geq \\psi(\\mu-\\nu)$ on a dense grid of parameter pairs spanning unbounded directions. Because $\\psi$ and $\\Psi$ are fixed convex radial functions, a single pair where the remainder violates either inequality for every admissible energy candidate refutes Proposition 3.3 for that objective, and with it the universality that Theorems 4.1, 4.2, and 5.1 assume. A cheaper probe is to move along a direction of negative curvature and measure whether $S_G(\\mu,\\nu)$ grows negatively without bound; if it does while $\\psi(0)$ must stay near zero, no convex radial lower bound can hold, and the framework's applicability to that network collapses.","supporting_citations":[],"review_version":1}