On the equivalence of a Hessian-free inequality and Lipschitz continuous Hessian

Bruno F. Louren\c{c}o; Minh N. Dao; Naoki Marumo; Radu I. Bo\c{t}; Tianxiang Liu

On the equivalence of a Hessian-free inequality and Lipschitz continuous Hessian

T0 review · 0 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The Hessian-free Jensen inequality exactly characterizes Lipschitz-continuous derivatives.

desk verdict Solid, honest resolution of the converse of the Hessian-free Jensen inequality; the proof via slicing and Baillon-Haddad is clean and the reflexivity assumption is exactly where it should be. read the letter →

arxiv 2504.17193 v2 pith:33G4CZ4K submitted 2025-04-24 math.OC math.FA

Radu I. Boţ , Minh N. Dao , Tianxiang Liu , Bruno F. Lourenço , Naoki Marumo This is my paper · ORCID

classification math.OCmath.FA MSC 49J5047H0590C2546B10

keywords Hessian-freeinequalityLipschitzcontinuousHessianJensen-typeBaillon-HaddadtheoremFréchetdifferentiabilityreflexiveBanachspacecocoercivityfirst-orderoptimization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper establishes a converse: a continuous map $F$ from a real Hilbert space $X$ into a real reflexive Banach space $Y$ has an $L$-Lipschitz Fréchet derivative if and only if it satisfies the Hessian-free Jensen inequality (1.2) with the same constant $L$, for every finite convex combination. The inequality involves only values of $F$ at finitely many points, never the Hessian or any second derivative. The proof works by slicing $F$ with dual functionals, applying convexity and the Baillon–Haddad theorem to each slice, and then gluing the slice derivatives back together using reflexivity. If correct, this turns a second-order regularity condition into a first-order, checkable condition and answers the natural question left open by the known forward implication from nonconvex optimization.

What carries the argument

The load-bearing object is inequality (1.2) in its two-point form, combined with the identity $t(1-t)\|x-y\|^2=(1-t)\|x\|^2+t\|y\|^2-\|x+t(y-x)\|^2$. This identity converts the two-sided bound on $\varphi_{y^*}$ into convexity of both quadratic perturbations $\frac{L}{2}\|\cdot\|^2-\varphi_{y^*}$ and $\frac{L}{2}\|\cdot\|^2+\varphi_{y^*}$. The Baillon–Haddad theorem — which identifies cocoercivity of a gradient with convexity of a quadratic perturbation — then yields Fréchet differentiability and cocoercivity of the perturbed gradients, giving the $L$-Lipschitz property for each slice; reflexivity of $Y$ is the bridge that assembles these slice derivatives into the derivative of $F$.

What would settle it

The decisive test is the two-point case with weights $1/2,1/2$. For $F(x)=|x|^\alpha$ on $\mathbb{R}$ with $0<\alpha<2$, taking $y=0$ gives $|F(x/2)-F(x)/2|=(2^{-\alpha}-1/2)|x|^\alpha$, which must be no larger than $\frac{L}{8}|x|^2$; letting $x\to 0$ shows no finite $L$ works, so non-differentiable Hölder maps are excluded. A continuous non-Fréchet-differentiable map from a Hilbert space to a reflexive Banach space that satisfies (1.2) for some $L$ would refute Theorem 1, and the scaling calculation shows why the inequality is strong enough to rule out the simplest nonsmooth candidates.

Watch

Extended reading notes

Core claim

The central claim is Theorem 1: for continuous $F$ and $L>0$, condition (i) — $F$ is Fréchet differentiable and $F'$ is $L$-Lipschitz — is equivalent to condition (ii), namely $\|F(\sum_i\lambda_i x_i)-\sum_i\lambda_i F(x_i)\|\le \frac{L}{2}\sum_{i<j}\lambda_i\lambda_j\|x_i-x_j\|^2$ holding for all convex weights. The proof of (ii)$\Rightarrow$(i) is the paper's contribution; (i)$\Rightarrow$(ii) is quoted from the known lemma. The proof shows each scalar slice $\varphi_{y^*}=y^*\circ F$ has an $L$-Lipschitz gradient, then uses reflexivity of $Y$ to define a candidate derivative $f_x(h)\in Y$ by $y^*(f_x(h))=\varphi'_{y^*}(x)h$, and identifies it as the Fréchet derivative of $F$ with the promised Lipschitz constant. In the scalar case this gives Corollary 1: a differentiable real-valued $f$ whose gradient satisfies (1.1) is twice differentiable with $L$-Lipschitz Hessian.

Load-bearing premise

The argument collapses if the target space $Y$ is not reflexive: the proof constructs the derivative at each point only by identifying a certain element of the second dual $Y^{**}$ with an actual vector in $Y$, and reflexivity is exactly the property that makes this identification valid.

Editorial extensions

If this is right

Every continuous map satisfying (1.2) is automatically Fréchet differentiable; no separate regularity assumption is needed.
The constant $L$ in the inequality is exactly the Lipschitz constant of the derivative, so the Hessian-free condition supplies quantitative information, not just qualitative smoothness.
For real-valued functions on Hilbert space, the gradient inequality (1.1) forces twice differentiability with $L$-Lipschitz Hessian, recovering the converse of the lemma used in first-order nonconvex optimization.
The equivalence holds for maps into every reflexive Banach space, so finite-dimensional intuition about the target space is not essential.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural test of sharpness is to drop reflexivity: if a continuous non-differentiable map from a Hilbert space into a non-reflexive space such as $c_0$ satisfied (1.2), the reflexivity assumption would be shown necessary; the paper's gluing step gives the precise place such an example would have to fail.
Because the proof checks only two-point convex combinations, the inequality could in principle be verified empirically on finitely many point pairs, yielding a computable lower bound on the Lipschitz constant of the derivative; algorithms might exploit this as a certificate, though the paper does not discuss this.
The same quadratic-perturbation technique suggests that analogues for higher-order smoothness would require a generalized Baillon–Haddad statement, which the paper's final remarks identify as a nontrivial obstacle.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this paper answers a real open question and does it honestly. The Hessian-free Jensen inequality (1.2) really does characterize Lipschitz differentiability for maps from Hilbert to reflexive Banach spaces. The converse is new; the forward direction was already in Marumo-Takeda. The result is meaningful for optimization theory, not earth-shaking, but solid.

What is new is the converse proof. The approach is to slice F with functionals y*, turn (1.2) into a two-sided convexity statement for the quadratic perturbations of each slice, apply a piece of Baillon-Haddad to get Lipschitz gradients on the slices, then assemble a Frechet derivative via reflexivity of Y to lift the Y**-valued object back to Y. That assembly step is the heart of the paper and it is well executed. The Hahn-Banach norm identity is used cleanly. The proof is reproducible from the text.

Soft spots, in proportion: the reflexivity assumption is essential, not hidden; the authors state it explicitly. But it means the vector-valued generalization stops exactly where the lifting argument needs it to stop. If Y is not reflexive, the derivative candidate lives in Y** and the argument does not deliver Frechet differentiability. That is a boundary, not a flaw. The forward direction is cited rather than re-derived; since it follows from the standard descent lemma plus a variance identity, the citation is reliable.

One minor quibble: the title and abstract talk about Lipschitz continuous Hessian, but the theorem concerns L-Lipschitz continuity of the derivative F'. For a vector-valued map between Banach spaces the Hessian is not defined in the usual sense. The scalar Corollary does give twice differentiability with Lipschitz Hessian, so the title is defensible, but the emphasis is slightly off.

The paper is for people working on smoothness structure in optimization, especially the line of work from Carmon et al. that uses Hessian smoothness through gradient inequalities. A reader who wants a precise characterization rather than folklore will get value.

Serious referee? Yes. The central argument holds up, the citations check out, and the result deserves to become a standard lemma. It needs no heavy revision.

Referee Report

0 major / 4 minor

Summary. The paper establishes Theorem 1, an equivalence between a Jensen-type inequality (1.2) and Lipschitz continuity of the Fréchet derivative for continuous maps from a Hilbert space into a reflexive Banach space. The forward direction is cited to Marumo–Takeda [7]; the paper's contribution is the converse. The converse proof has two parts: Lemma 2 uses the n=2 case of (1.2) and the Baillon–Haddad theorem to show that every unit-ball slice y*∘F has an L-Lipschitz continuous gradient; Lemma 3 shows, via reflexivity of the codomain, that these slices assemble into a Fréchet derivative of F itself with the same Lipschitz constant. Corollary 1 yields the converse of Lemma 1: a differentiable scalar function whose gradient satisfies (1.1) is twice differentiable with L-Lipschitz Hessian.

Significance. The result is clean and exact: the constant L is preserved and the characterization is genuinely Hessian-free. The proof is transparent and carefully handles the infinite-dimensional codomain, with reflexivity explicitly used to lift the derivative from Y** to Y. The paper thus clarifies why reflexivity is the natural hypothesis and strengthens the known forward direction into a full equivalence. The use of the Baillon–Haddad theorem is appropriate, the derivation is verifiable, and I found no circularity. This is a worthwhile contribution to the optimization literature on Hessian-free methods.

minor comments (4)

[Theorem 1 (Section 1)] The forward implication (i)⇒(ii) is not proved in the manuscript; the text after Theorem 1 only states that the proof is essentially the same as [7, Lemma 3.1]. Since (i)⇒(ii) is part of the stated equivalence, please add a short proof—for example, apply the standard descent inequality |φ(y)−φ(x)−φ'(x)(y−x)| ≤ L/2 ||y−x||² to each slice φ_y* = y*∘F and use the identity Σ_i λ_i ||x_i − Σ_j λ_j x_j||² = Σ_{i<j} λ_i λ_j ||x_i − x_j||²—so that the paper is self-contained.
[Lemma 2] The sentence beginning "In particular, we have L||·||² − (L/2||·||² + φ_y*) = L/2||·||² − φ_y* convex" is potentially misleading, since a difference of two convex functions is not convex in general. The intended statement is that L/2||·||² − φ_y*, which was already proved convex, is the function to which Theorem 2 applies with β = 2L; please rephrase to avoid the appearance of an invalid inference.
[Lemma 3, part (3)] In the displayed chain for ||fx − fy||, the equality between sup_{||h||≤1, y*∈Y*_1} y*(fx(h) − fy(h)) and sup_{y*∈Y*_1} ||φ'_y*(x) − φ'_y*(y)|| uses the fact that sup_{||h||≤1} y*((Tx − Ty)h) = ||y*∘(Tx − Ty)|| and that the supremum over the unit ball of Y* of these norms is the operator norm; please state this briefly to help the reader.
[General] Please fix typographical errors: "continuos" in the abstract, "Lipschi tz" in the title/first line, and the garbled accents in "Fr´echet" and "H¨ older" in the references.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the converse is proved from external Baillon-Haddad and Hahn-Banach theorems, and the only self-citation is a standard forward-direction lemma that is not the paper's contribution.

full rationale

The central new result is (ii) implies (i) in Theorem 1. Lemma 2 derives from (1.2) with n=2 the estimates (2.3), which make L/2||x||^2 - phi_{y*} and L/2||x||^2 + phi_{y*} convex; applying the external Baillon-Haddad theorem (Bauschke and Combettes [2, Theorem 2.1]) to g = L/2||x||^2 + phi_{y*} yields cocoercivity of L Id + nabla phi_{y*} and hence the L-Lipschitz continuity of nabla phi_{y*}. Lemma 3 is a direct Banach-space construction: each map y* maps to phi'_{y*}(x) is bounded and linear, so reflexivity of Y identifies the map y* maps to phi'_{y*}(x)h with an element fx(h) of Y; Hahn-Banach then provides the norm estimates and proves F is Frechet differentiable with L-Lipschitz derivative. None of these steps assumes the conclusion of Theorem 1. The only self-citation is the forward implication (i) implies (ii), attributed to Marumo and Takeda [7, Lemma 3.1]; the paper says 'The proof of (i) = implies (ii) is essentially the same as that of Lemma 1', and this direction is not the claimed new contribution. The cited lemma is independently published and its validity does not depend on the present theorem, so it is not load-bearing circularity. Reflexivity of Y is an explicit hypothesis, not a hidden assumption, and no fitted parameter or post-hoc adjustment appears. Thus the derivation is self-contained and no circular step can be exhibited.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The proof introduces no new entities or parameters beyond the given constant L. It relies on classical theorems (Baillon-Haddad, Hahn-Banach) and on the cited forward direction from [7]. Reflexivity of Y is an explicit structural assumption.

assumptions (4)

standard math Baillon-Haddad theorem as stated in [2, Theorem 2.1]: for proper, convex, lower semicontinuous g on a Hilbert space, β/2||·||²-g convex iff g is Frechet differentiable with 1/β-cocoercive gradient.
Quoted as Theorem 2 and used to pass from convexity of L/2||·||²+φ to differentiability and cocoercivity in Lemma 2.
standard math The forward implication (i)⇒(ii): an L-smooth function satisfies inequality (1.2), as established in [7, Lemma 3.1].
Stated in the introduction and used to assert the equivalence in Theorem 1; the proof is not reproduced.
standard math Hahn-Banach theorem: for every y in a Banach space, ||y|| = sup_{||y*||≤1} y*(y).
Used in Lemma 3 to compute norms of values of F and of the difference of derivatives.
domain assumption Reflexivity of Y: every element of Y** is the evaluation functional of an element of Y.
Used in Lemma 3 to construct fx(h) ∈ Y from the Y**-valued map y* ↦ φ'_{y*}(x)h; the theorem assumes Y reflexive.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On the equivalence of a Hessian-free inequality and Lipschitz continuous Hessian." pith.science (2026). https://pith.science/paper/33G4CZ4K

@misc{pith2026250417193,
  author       = {Pith},
  title        = {Pith review of: On the equivalence of a Hessian-free inequality and Lipschitz continuous Hessian},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/33G4CZ4K}},
  note         = {Machine review of arXiv:2504.17193}
}

read the original abstract

It is known that if a twice differentiable function has a Lipschitz continuous Hessian, then its gradients satisfy a Jensen-type inequality. In particular, this inequality is Hessian-free in the sense that the Hessian does not actually appear in the inequality. In this paper, we show that the converse holds in a generalized setting: if a continuos function from a Hilbert space to a reflexive Banach space satisfies such an inequality, then it is Fr\'echet differentiable and its derivative is Lipschitz continuous. Our proof relies on the Baillon-Haddad theorem.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

11 extracted references · 7 canonical work pages

[7]

Marumo and A

N. Marumo and A. Takeda. Parameter-free accelerated gra dient descent for nonconvex minimization. SIAM Journal on Optimization , 34(2):2093–2120, 2024. doi:10.1137/22M1540934

doi:10.1137/22m1540934 2024
[1]

Allen-Zhu and Y

Z. Allen-Zhu and Y. Li. NEON2: Finding local minima via ﬁr st-order oracles. In S. Ben- gio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi , and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc.,

work page
[2]

H. H. Bauschke and P. L. Combettes. The Baillon-Haddad th eorem revisited. Journal of Convex Analysis , 17(3&4):781–787, 2010

work page 2010
[3]

Convex until proven guilty

Y. Carmon, J. C. Duchi, O. Hinder, and A. Sidford. “Convex until proven guilty”: Dimension-free acceleration of gradient descent on non-co nvex functions. In D. Precup and Y. W. Teh, editors, Proceedings of the 34th International Conference on Machine 6 Learning, volume 70 of Proceedings of Machine Learning Research, pages 654–663. PMLR, 06–11 Aug 2017. U...

work page 2017
[4]

J. B. Conway. A Course in Functional Analysis . Graduate Texts in Mathematics. Springer New York, 2007

work page 2007
[5]

C. Jin, P. Netrapalli, and M. I. Jordan. Accelerated grad ient descent escapes saddle points faster than gradient descent. In S. Bubeck, V. Perche t, and P. Rigollet, editors, Proceedings of the 31st Conference On Learning Theory , volume 75 of Proceedings of Machine Learning Research, pages 1042–1085. PMLR, 06–09 Jul 2018

work page 2018
[6]

Li and Z

H. Li and Z. Lin. Restarted nonconvex accelerated gradie nt descent: No more poly- logarithmic factor in the O(ǫ−7/ 4) complexity. Journal of Machine Learning Research , 24(157):1–37, 2023. URL: http://jmlr.org/papers/v24/22-0522.html

work page 2023
[8]

Marumo and A

N. Marumo and A. Takeda. Universal heavy-ball method for nonconvex op- timization under H¨ older continuous Hessians. Mathematical Programming , 2024. doi:10.1007/s10107-024-02100-4

doi:10.1007/s10107-024-02100-4 2024

Show all 11 references

[9]

Wachsmuth and G

D. Wachsmuth and G. Wachsmuth. A simple proof of the Baill on-Haddad theorem on open subsets of Hilbert spaces. arXiv e-print , 2022. arXiv:2204.00282

2022 arXiv
[10]

Y. Xu, R. Jin, and T. Yang. NEON+: Accelerated gradient m ethods for extracting negative curvature for non-convex optimization. arXiv e-print, 2017. arXiv:1712.01033. 7

2017 arXiv
[2018]

URL: https://papers.nips.cc/paper/by-source-2018-1873

2018

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [7]

Marumo and A

N. Marumo and A. Takeda. Parameter-free accelerated gra dient descent for nonconvex minimization. SIAM Journal on Optimization , 34(2):2093–2120, 2024. doi:10.1137/22M1540934

doi:10.1137/22m1540934 2024

[2] [1]

Allen-Zhu and Y

Z. Allen-Zhu and Y. Li. NEON2: Finding local minima via ﬁr st-order oracles. In S. Ben- gio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi , and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc.,

work page

[3] [2]

H. H. Bauschke and P. L. Combettes. The Baillon-Haddad th eorem revisited. Journal of Convex Analysis , 17(3&4):781–787, 2010

work page 2010

[4] [3]

Convex until proven guilty

Y. Carmon, J. C. Duchi, O. Hinder, and A. Sidford. “Convex until proven guilty”: Dimension-free acceleration of gradient descent on non-co nvex functions. In D. Precup and Y. W. Teh, editors, Proceedings of the 34th International Conference on Machine 6 Learning, volume 70 of Proceedings of Machine Learning Research, pages 654–663. PMLR, 06–11 Aug 2017. U...

work page 2017

[5] [4]

J. B. Conway. A Course in Functional Analysis . Graduate Texts in Mathematics. Springer New York, 2007

work page 2007

[6] [5]

C. Jin, P. Netrapalli, and M. I. Jordan. Accelerated grad ient descent escapes saddle points faster than gradient descent. In S. Bubeck, V. Perche t, and P. Rigollet, editors, Proceedings of the 31st Conference On Learning Theory , volume 75 of Proceedings of Machine Learning Research, pages 1042–1085. PMLR, 06–09 Jul 2018

work page 2018

[7] [6]

Li and Z

H. Li and Z. Lin. Restarted nonconvex accelerated gradie nt descent: No more poly- logarithmic factor in the O(ǫ−7/ 4) complexity. Journal of Machine Learning Research , 24(157):1–37, 2023. URL: http://jmlr.org/papers/v24/22-0522.html

work page 2023

[8] [8]

Marumo and A

N. Marumo and A. Takeda. Universal heavy-ball method for nonconvex op- timization under H¨ older continuous Hessians. Mathematical Programming , 2024. doi:10.1007/s10107-024-02100-4

doi:10.1007/s10107-024-02100-4 2024