Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Learning Parametric Convex Functions

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A neural network architecture with nonnegative skip-layer weights and convex activations produces parametrized convex functions that can be fit to data and used directly in convex optimization.

desk verdict A clean, practical extension of ICNNs to parametric convex functions; the core proof is sound, but the quadratic experiment's sampling description and the monotonicity prose need fixing. read the letter →

arxiv 2506.04183 v1 pith:7XLD6UQJ submitted 2025-06-04 math.OC

Maximilian Schaller , Alberto Bemporad , Stephen Boyd This is my paper
classification math.OC MSC 90C2568T07
keywords parametrizedconvexfunctionsinputneuralnetworksdisciplinedprogrammingoptimizationfunctionfittingapproximatedynamicbatteryaginghypernetwork
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a neural network architecture that learns a parametrized convex function (PCF) from data: a function $f(x,\theta)$ that is convex in $x$ for every parameter value $\theta$. The architecture generalizes input-convex neural networks by letting a second network produce all weights and offsets as functions of $\theta$, while enforcing elementwise nonnegativity on the skip-layer weight matrices. Because the construction follows the composition rule of disciplined convex programming, the learned expression can be exported directly to a domain-specific language for convex optimization and used as a building block in parametrized convex optimization problems. The authors demonstrate the method on synthetic examples, a battery aging model, and an approximate dynamic programming controller. If the approach holds up, convex optimization problems can be partly data-driven while retaining first-principles convex structure.

What carries the argument

The load-bearing object is the parameter network $\psi(\theta)$, a hypernetwork that takes $\theta$ as input and emits all layer weights, input-skip weights, and offsets of the main network. Its output entries for the matrices $W^l(\theta)$, $l=2,\dots,L$, are passed through a nonnegative activation such as ReLU or logistic, which is what guarantees convexity in $x$ for every $\theta$. The convexity proof is the composition rule from convex analysis: a nondecreasing convex function composed with a nonnegative weighted sum of convex functions plus an affine term remains convex. This is the same rule that disciplined convex programming uses, which is why the fitted expression can be translated into a modeling-language expression for convex optimization.

What would settle it

Train the architecture on data from $f(x,\theta)=x^T Q(\theta)x$ with $Q(\theta)$ a positive semidefinite matrix whose entries oscillate at increasing frequency in $\theta$; if the test error grows without bound as the frequency increases while the training error stays small, the hypernetwork's expressiveness is insufficient for that family.

Watch

Extended reading notes

Core claim

The central claim is that the architecture defined by $z_0=x$, $z_l=\phi(W^l(\theta)z_{l-1}+V^l(\theta)x+\omega^l(\theta))$ for $l=1,\dots,L-1$, and $y=W^L(\theta)z_{L-1}+V^L(\theta)x+\omega^L(\theta)$, with all $W^l(\theta)$ elementwise nonnegative for $l=2,\dots,L$ and $\phi$ a nondecreasing convex activation, produces a parametrized convex function for any $\theta$. The argument is a recursion: nonnegative weighted sums of convex functions are convex, and the composition rule preserves convexity, exactly as in disciplined convex programming. Thus the model is a PCF by construction and is DCP-expressible. Fitting the network weights by minimizing a regularized average loss therefore yields a data-driven PCF that can be used in convex optimization formulations.

Load-bearing premise

The method assumes that the secondary network that generates the weights can represent the weight matrices accurately enough over the entire parameter range, with no guarantee that this is possible; success depends on the chosen architecture and on the optimizer finding a good fit.

Editorial extensions

If this is right

  • A fitted PCF can be exported to a disciplined convex programming expression and placed anywhere a convex expression is allowed, such as an objective or a constraint, enabling parametrized convex optimization families built from data.
  • The same architecture supports extensions: adding a convex quadratic term $x^T Q(\theta)x$, enforcing monotonicity in $x$, and encouraging a prescribed minimizer via a subgradient penalty.
  • The framework also fits parametrized convex sets by learning a PCF level-set boundary $\{x : f(x,\theta)\le 0\}$ from membership data.
  • In the battery aging example, the learned PCF achieves about seven times lower test root-mean-square error than the short-term convex approximation it is compared against.
  • In approximate dynamic programming, the learned PCF serves as a convex surrogate value function, so the one-step lookahead control policy solves a convex problem at each time step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not state this, but the architecture's differentiability with respect to $x$ means the same training framework could also supervise the gradient of the PCF directly, which would be useful for value functions in control where gradient information is readily available.
  • A testable extension is to apply the method to higher-dimensional variables and parameters; the current experiments are low-dimensional, and performance may degrade when the hypernetwork must interpolate many weight matrices across a large $\Theta$.
  • The paper's expressiveness assumption points to a concrete research direction: proving function-approximation or sample-complexity bounds for this parameterized input-convex network class, which would tell practitioners how large the hypernetwork must be for a target accuracy.
  • Another consequence not developed in the paper is that the fitted functions inherit the atom library of disciplined convex programming, so the same learned expression could be composed with existing atoms to build interpretable convex models without retraining.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper addresses the problem of fitting a parametrized convex function (PCF) to data, i.e., a function f(x, θ) that is convex in x for every allowed θ and is expressible in a disciplined convex programming (DCP) language. The proposed architecture (Eqs. 3–4) generalizes input-convex neural networks: the layer weights, offsets, and optional quadratic term are generated by a hypernetwork ψ(θ), and convexity is enforced by requiring the forward weight matrices W^l(θ) to be elementwise nonnegative and the activation to be nondecreasing and convex. The authors implement this in the open-source package LPCF, describe extensions for quadratic terms, monotonicity, subgradient specification, and convex-set fitting, and report experiments on a univariate piecewise-affine function, a quadratic function on R^3, a battery-aging model, and an approximate-dynamic-programming pendulum example. The central convexity argument is correct and DCP-compatible, and the paper offers a clean path from data to reusable convex expressions in CVXPY.

Significance. If the empirical claims hold, the paper provides a useful and practical tool: a data-driven way to obtain DCP-expressible PCFs that can be inserted directly into convex optimization families. The core mathematical construction in Section 2.1 is sound; the composition argument using nonnegative W^l and nondecreasing convex activations is exactly the standard DCP rule. The open-source implementation, the reproducible experimental pipeline, and the honest comparison with a nonparametric baseline in Section 4.1 are clear strengths. The main weakness is that the paper offers no expressiveness or sample-complexity analysis for the combined ICNN-plus-hypernetwork class, and one of the four experiments (Section 4.2) has a data-generation flaw that weakens the validation. These issues are fixable and do not undermine the convexity proof, but they do affect the strength of the paper's central practical claim.

major comments (3)
  1. [Section 4.2] The quadratic experiment is described as generating f_true(x, θ) = x^T θ x with θ ∈ S^3_+ (the positive semidefinite cone), which is convex in x. However, the experimental setup states that each entry of θ is drawn uniformly from [-1, 1]; a random symmetric matrix with entries in [-1, 1] is not generally positive semidefinite, so many of the sampled instances are not convex in x. The reported RMSE therefore does not specifically demonstrate learning a convex family; it conflates approximation error with the effect of projecting nonconvex data onto the convex class. Please either sample θ in a way that guarantees PSD (e.g., θ = U^T U with random entries in U) and report the convex subset, or explicitly reframe the experiment as fitting the best convex approximation and remove the PSD claim.
  2. [Section 2.2 and Section 4] The paper's central practical claim of a 'seamless path from data to a PCF' rests on the hypernetwork ψ being able to represent the weight matrices W^l(θ), V^l(θ), and offsets with sufficient accuracy over the parameter range Θ. Section 2.2 gives default hyperparameters and training heuristics, but provides no expressiveness result, no sample-complexity bound, and no capacity study for the combined ICNN-plus-hypernetwork architecture. If ψ is too small, or if the nonconvex fitting problem is not solved well, the learned f may be far from the target PCF. This is a genuine gap in the analysis; I ask the authors to either add a discussion of the approximation limitations or provide an ablation varying the widths of ψ and reporting train/test RMSE, or both.
  3. [Section 3.2] The monotone-decreasing construction is stated as 'cascading instead V^j with a nonpositive activation function φ−, such as φ− = −φ+', which as written suggests replacing the activation φ in Eq. (3) by −ReLU or −softplus. Such a replacement would destroy convexity, since −ReLU is not convex. The intended construction is presumably to make the entries of V^j nonpositive while keeping φ nondecreasing and convex, in which case the stated conclusion (convex and decreasing) is correct. Please rewrite this paragraph to remove the ambiguity and avoid presenting a false construction.
minor comments (4)
  1. [Section 1.4] In the paragraph on non-parametric PCF fitting, the phrase 'when p > 0 and r = 0' appears to be a typo; the preceding sentence already treats p > 0, and the regularizer r was not introduced in that context. Please clarify the intended condition.
  2. [Figure 2] The code snippet in Figure 2 contains a misspelling in the comment 'inv ol vi ng f' and in the variable name 'P ar am et er'; these should be corrected for readability.
  3. [Section 4.3] The experimental setup says 'We generate 1000 values of θ, with accumulated charge throughput A ∈ [0, 50] and temperature T ∈ [10, 50]' but then notes that Q = 1 is fixed. Since θ = (A, Q, T), the description should state explicitly that Q is not sampled.
  4. [References] Reference [Kin14] is titled 'ADAN', which appears to be a typo for 'ADAM'. Please correct it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the convexity proof is self-contained and the fitting experiments use independently generated data.

full rationale

The paper's central claim is that the architecture (3)-(4), with elementwise nonnegative W^l(theta) for l=2,...,L and nondecreasing convex activations, yields a parametrized convex function. This is established in Section 2.1 by an induction argument using the standard composition rule for convex functions, cited to Boyd and Vandenberghe. The argument does not rely on any fitted parameter, on data, or on self-citation; it is a direct mathematical proof from the stated architecture assumptions. The fitting procedure in Section 1.4 is standard regularized empirical risk minimization, and the experiments in Section 4 evaluate the fitted PCF against independently generated data: a piecewise affine function, a quadratic function, a battery aging model, and a pendulum optimal control problem. Baselines such as the short-term battery approximation from [NOBL25] are external comparators, not premises of the convexity claim. The self-citations to prior work by the authors (e.g., [BAB20], [NOBL25]) provide background or baselines and are not load-bearing for the derivation. The limitation noted in Section 1.4 that the unregularized non-parametric solution is likely to generalize poorly is a caveat about that baseline method, not a circular step. The inconsistency in Section 4.2, where theta is said to lie in S^3_+ but entries are sampled uniformly in [-1,1], weakens that particular experiment's validity, but it is a data-generation bug rather than a fitted-input-called-prediction or self-definitional circularity. No step in the paper reduces to its own inputs by construction.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central convexity guarantee relies only on standard convex composition rules and the choice of nondecreasing convex activations. The experiments depend on typical deep-learning assumptions (representational capacity, optimizer success) that are not proven. No free parameters are needed to establish the convexity claim itself; hyperparameters such as layer widths and regularization are user choices.

free parameters (2)
  • Regularization hyperparameter lambda = 0 by default; elastic net coefficients 10^-8 (quadratic) and 0.1 (L1) in Section 4.4
    Scales the regularizer in the training loss (2); chosen by hand, not fitted to data, and does not affect the convexity guarantee.
  • Subgradient regularization weight rho_min = not reported
    Scales the gradient penalty in (5), used in Section 4.4; its value is not stated, making the reported result hard to reproduce exactly.
assumptions (4)
  • standard math Composition rule for convex functions: if phi is nondecreasing and convex, then phi composed with a convex function is convex.
    Used in Section 2.1 to prove each layer z_l is convex in x.
  • domain assumption The chosen activations (ReLU, softplus) are nondecreasing and convex.
    Required for the convexity argument and for the monotonicity extension in Section 3.2.
  • domain assumption The hypernetwork psi(theta) is sufficiently expressive and trainable to represent the required weight mappings.
    No approximation-theoretic guarantee is provided; success relies on standard neural-network expressiveness and optimizer performance (Section 2.2).
  • standard math Nonnegative linear combinations of convex functions, plus affine terms, are convex.
    The central convexity proof in Section 2.1 applies this repeatedly.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning Parametric Convex Functions." pith.science (2026). https://pith.science/paper/7XLD6UQJ

@misc{pith2026250604183,
  author       = {Pith},
  title        = {Pith review of: Learning Parametric Convex Functions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7XLD6UQJ}},
  note         = {Machine review of arXiv:2506.04183}
}
read the original abstract

A parametrized convex function depends on a variable and a parameter, and is convex in the variable for any valid value of the parameter. Such functions can be used to specify parametrized convex optimization problems, i.e., a convex optimization family, in domain specific languages for convex optimization. In this paper we address the problem of fitting a parametrized convex function that is compatible with disciplined programming, to some given data. This allows us to fit a function arising in a convex optimization formulation directly to observed or simulated data. We demonstrate our open-source implementation on several examples, ranging from illustrative to practical.

Figures

Figures reproduced from arXiv: 2506.04183 by the authors.

Figure 1
Figure 1. Neural network architecture for PCF y = f(x, θ). is a convex function of x, since it is a nonnegative weighted sum of convex functions, plus an affine function of x. By the composition rule [BV04, §3.2.4], each entry of z l is a convex function of x. This argument is exactly the one used in DCP. This means that, assuming the activation function is an atom, f(x, θ) is DCP. As in an input convex neural network, our ar… view at source ↗
Figure 2
Figure 2. Using LPCF with CVXPY. The initialization code for dimensions n, p, data Y, X, Theta, and CVXPY objects g and constraints is omitted for clarity. CVXPY problem in line 15. In this example we simply add the fitted PCF to the objective, but we note that it can be used anywhere in CVXPY that a convex function appear, e.g., in constraints. To fit the weights w of the network ψ we use jax-sysid [Bem24], a Python package … view at source ↗
Figure 3
Figure 3. Neural network architecture for ψ(θ). of Adam [Kin14] to obtain a good set of network weights, followed by 2000 iterations of L-BFGS-B [BLNZ95] (a variant of L-BFGS for bound-constrained optimization, to respect the nonnegativity constraints on Wl ) to refine the model weights. We run the entire fitting method on multiple random initial sets of model weights, and take the best one as our final choice. We parallelize… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Data-generating function f true and learned PCF f for four parameter values. Top. Convex f true . Bottom. Non-convex f true . 13 [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Data-generating function f true and learned PCF f for three parameter values. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Left: ADP control input ˆu0 versus nonlinear control input u ⋆ 0 . Right: Simulation of the two controllers starting from z0 = (0, 0), where θ = m = 1. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Estimating Dynamic Marginal Policy Effects under Sequential Unconfoundedness

    stat.ME 2026-04 unverdicted novelty 7.0 of 10

    Parametric nonconvex problems can be approximated by a min of quasiconvex (convex∘monotonic) surrogates and solved via parallel convex optimization, trained mainly from function evaluations.

Reference graph

Works this paper leans on

33 extracted references · 21 canonical work pages · cited by 1 Pith paper

  1. [1]

    Agrawal, R

    A. Agrawal, R. Verschueren, S. Diamond, and S. Boyd. A rewriting system for convex optimization problems. Journal of Control and Decision , 5(1):42--60, 2018

    2018

  2. [2]

    B. Amos, L. Xu, and Z. Kolter. Input convex neural networks. In International conference on machine learning , pages 146--155. PMLR, 2017

    work page 2017

  3. [3]

    Abdufattokhov, M

    S. Abdufattokhov, M. Zanon, and A. Bemporad. Learning Lyapunov terminal costs from data for complexity reduction in nonlinear model predictive control. Int. J. Robust Nonlinear Control , 34(13):8676--8691, 2024

    work page 2024

  4. [4]

    S. Boyd, A. Agrawal, and S. Barratt. Embedded convex optimization for control. In Proceedings 59th IEEE Conference on Decision and Control . IEEE, 2020

    work page 2020

  5. [5]

    An L-BFGS-B approach for linear and nonlinear system identification under $\ell_1$ and group-Lasso regularization

    A. Bemporad. Linear and nonlinear system identification under _1 - and group- Lasso regularization via L-BFGS-B . Submitted for publication , 2024. Also available on arXiv at http://arxiv.org/abs/2403.03827

    work page Pith review arXiv 2024

  6. [6]

    Bertsekas

    D. Bertsekas. Dynamic Programming and Optimal Control , volume 2. Athena Scientific, Belmont, MA, 3 edition, 2005

    work page 2005

  7. [7]

    Bradbury, R

    J. Bradbury, R. Frostig, P. Hawkins, M. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. Vander P las, S. Wanderman- M ilne, and Q. Zhang. JAX : C omposable transformations of P ython+ N um P y programs, 2018. Available at http://github.com/jax-ml/jax

    work page 2018

  8. [8]

    R. Byrd, P. Lu, J. Nocedal, and C. Zhu. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing , 16(5):1190--1208, 1995

    work page 1995

Show all 33 references
  1. [9]

    Boyd and L

    S. Boyd and L. Vandenberghe. Convex Optimization . Cambridge University Press, 2004

    2004

  2. [10]

    Diamond and S

    S. Diamond and S. Boyd. CVXPY : A P ython-embedded modeling language for convex optimization. Journal of Machine Learning Research , 17(83):1--5, 2016

    2016

  3. [11]

    Dunning, J

    I. Dunning, J. Huchette, and M. Lubin. JuMP : A modeling language for mathematical optimization. SIAM Review , 59(2):295--320, 2017

    2017

  4. [12]

    N. Draper. Applied regression analysis . McGraw-Hill. Inc, 1998

    1998

  5. [13]

    Deschatre and X

    T. Deschatre and X. Warin. Input convex K olmogorov A rnold networks. arXiv preprint arXiv:2505.21208 , 2025

    2025

  6. [14]

    Ebbesen, P

    S. Ebbesen, P. Elbert, and L. Guzzella. Battery state-of-health perceptive energy management for hybrid electric vehicles. IEEE Transactions on Vehicular technology , 61(7):2893--2900, 2012

    2012

  7. [15]

    A. Fu, B. Narasimhan, and S. Boyd. CVXR : A n R package for disciplined convex optimization. Journal of Statistical Software , 94(14):1--34, 2020

    2020

  8. [16]

    Grant and S

    M. Grant and S. Boyd. CVX : M atlab software for disciplined convex programming, version 2.1, 2014

    2014

  9. [17]

    Garcia, D

    C. Garcia, D. Prett, and M. Morari. Model predictive control: T heory and practice -- a survey. Automatica , 25(3):335--348, 1989

    1989

  10. [18]

    Huber and E

    P. Huber and E. Ronchetti. Robust statistics . John Wiley & Sons, 2011

    2011

  11. [19]

    P. Huber. Robust estimation of a location parameter. In Breakthroughs in statistics: M ethodology and distribution , pages 492--518. Springer, 1992

    1992

  12. [20]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770--778. IEEE, 2016

    2016

  13. [21]

    K. He, X. Zhang, S. Ren, and J. Sun. Identity mappings in deep residual networks. In Computer Vision--ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11--14, 2016, Proceedings, Part IV 14 , pages 630--645. Springer, 2016

    2016

  14. [22]

    Kouvaritakis and M

    B. Kouvaritakis and M. Cannon. Model predictive control . Springer, 2016

    2016

  15. [23]

    D. Kingma. ADAN : A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014

    2014 arXiv

  16. [24]

    L\"ofberg

    J. L\"ofberg. YALMIP : A toolbox for modeling and optimization in Matlab . In IEEE International Conference on Robotics and Automation (ICRA) , pages 284--289. IEEE, 2004

    2004

  17. [25]

    Liu and F

    Y. Liu and F. Oliveira. ICNN -enhanced 2SP : Leveraging input convex neural networks for solving two-stage stochastic programming. arXiv preprint arXiv:2505.05261 , 2025

    2025 arXiv

  18. [26]

    Nnorom, G

    O. Nnorom, G. Ogut, S. Boyd, and P. Levis. Aging-aware battery control via convex optimization. arXiv preprint arXiv:2505.09030 , 2025

    2025 arXiv

  19. [27]

    K. Pearson. On the general theory of skew correlation and non-linear regression . Dulau and Company, 1905

    1905

  20. [28]

    Rawlings, D

    J. Rawlings, D. Mayne, M. Diehl, et al. Model predictive control: T heory, computation, and design , volume 2. Nob Hill Publishing Madison, WI, 2017

    2017

  21. [29]

    Suri and S

    G. Suri and S. Onori. A control-oriented cycle-life model for hybrid electric vehicle lithium-ion batteries. Energy , 96:644--653, 2016

    2016

  22. [30]

    Serrao, S

    L. Serrao, S. Onori, A. Sciarretta, Y. Guezennec, and G. Rizzoni. Optimal energy management of hybrid electric vehicles including battery aging. In Proceedings of the 2011 American control conference , pages 2125--2130. IEEE, 2011

    2011

  23. [31]

    Stellato

    B. Stellato. Mixed-integer optimal control of fast dynamical systems . PhD thesis, University of Oxford, 2017

    2017

  24. [32]

    Udell, K

    M. Udell, K. Mohan, D. Zeng, J. Hong, S. Diamond, and S. Boyd. Convex optimization in J ulia. SC14 Workshop on High Performance Technical Computing in Dynamic Languages , 2014

    2014

  25. [33]

    Zou and T

    H. Zou and T. Hastie. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society Series B: S tatistical Methodology , 67(2):301--320, 2005

    2005

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.