Pith. sign in

REVIEW 3 major objections 5 minor 42 references

Convexity in ReLU Neural Networks: beyond ICNNs?

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper derives necessary and sufficient conditions for a ReLU neural network to implement a convex function, and uses them to show that Input Convex Neural Networks are exactly the convex one-hidden-layer ReLU networks but strictly…

desk verdict A solid theoretical contribution on convex ReLU networks with a clean one-hidden-layer result and a nice counterexample, but the sufficiency theorem and the 'exact' numerical checker rest on a genericity assumption that is asserted, not proven. read the letter →

arxiv 2501.03017 v2 pith:3EA4NGPR submitted 2025-01-06 cs.LG

Anne Gagneux , Mathurin Massias , Emmanuel Soubies , Rémi Gribonval This is my paper · ORCID
classification cs.LG MSC 68T0726B25
keywords convexneuralnetworksReLUInputpiecewiselinearfunctionspath-liftingconvexitycharacterizationDAGarchitecturesexactcheck
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether standard Input Convex Neural Networks, which enforce convexity by clipping hidden-layer weights to non-negative values, cover every convex function a given ReLU architecture can express. For one hidden layer the answer is yes: any convex one-hidden-layer ReLU network can be rewritten as an ICNN of the same width. From two hidden layers onward the answer is no: the paper exhibits a small convex network that no ICNN with the same architecture can implement. The engine is an exact characterization: under a genericity condition, a DAG ReLU network is convex if and only if, for every hidden neuron, a certain inner product of its outgoing path weights and reachable path activations is non-negative. This turns convexity into a finite check and yields an exact numerical test.

What carries the argument

The path-lifting formalism: each path from a neuron $\nu$ to the output contributes a product of weights, and its activation is the product of neuron activations along the path, so the network output is written as a scalar product between the path-lifting vector and the path-activations matrix. Convexity is then checked at points $X_\nu$ where only neuron $\nu$ switches: local monotonicity of the gradient across such a switch is equivalent to $\langle a^{\nu\to}, \Phi^{\nu\to}\rangle \ge 0$. This is combined with a minimal characterization of convex continuous piecewise-linear functions, Proposition 3.7, which says convexity only needs to be tested at pairs of points around the frontiers between neighboring affine regions.

What would settle it

Search small random two-hidden-layer networks while explicitly verifying Assumption 7.1; if any network satisfies $\min_{x\in X_\nu} \langle a^{\nu\to}, \Phi^{\nu\to}\rangle \ge 0$ for every hidden neuron $\nu$ yet has a non-convex restriction along some line segment, Theorem 7.4's sufficiency direction is false. Alternatively, run Algorithm 1 on a network with two neurons switching at the same frontier point and check whether it still reports convexity; if it does, the unverified assumption is doing real work in the test.

Watch

Extended reading notes

Core claim

The central discovery is a necessary and sufficient condition for convexity of deep ReLU networks, stated via path-lifting. For a DAG ReLU network whose hidden neurons are isolated and whose slope-changing frontiers involve a single switching neuron, the function $f_\theta$ is convex if and only if for every hidden neuron $\nu$ one has $\min_{x\in X_\nu} \langle a^{\nu\to}(x,\theta), \Phi^{\nu\to}(\theta)\rangle \ge 0$, where $\Phi^{\nu\to}(\theta)$ collects the products of weights along paths from $\nu$ to the output and $a^{\nu\to}(x,\theta)$ records which of those paths are active at $x$. The paper also proves that for one hidden layer this condition reduces to non-negativity of the output weights, i.e. to ICNN; for two hidden layers it permits non-ICNN convex networks, exemplified by the explicit function $f_{\text{EX}}$ that no same-architecture ICNN can implement.

Load-bearing premise

The characterization's sufficiency and the numerical algorithm rely on Assumption 7.1: every hidden neuron is isolated, and at every point where the function's slope changes only one neuron's activation actually switches; the paper argues this is generic but does not prove it, and Algorithm 1 does not verify it before declaring convexity.

Editorial extensions

If this is right

  • For one-hidden-layer ReLU networks, the ICNN restriction is not a loss of expressive power: every convex function from such a network can be re-expressed as an ICNN with the same width.
  • With two hidden layers, ICNNs are strictly less expressive than unconstrained ReLU networks: the paper constructs a convex function realizable by a two-hidden-layer, two-neuron-per-layer network that no ICNN with the same architecture can realize.
  • For general DAG ReLU networks, under a generic non-degeneracy assumption, convexity is equivalent to finitely many non-negativity constraints on inner products of path-liftings and reachable path-activations.
  • These constraints give an exact numerical algorithm that checks convexity of moderate-size ReLU networks by extracting their polyhedral complex, rather than by sampling or approximation.
  • Random Gaussian draws show that convex ReLU networks become increasingly more common than ICNNs as width grows, indicating that ICNN constraints substantially narrow the set of convex functions reachable by a fixed architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If Assumption 7.1 is indeed generic as the authors argue, the same finite inequalities could serve as differentiable penalties during training, turning convexity into a soft constraint that does not require architectural weight clipping.
  • The path-based inequality may generalize to other structural properties of learned operators, such as monotonicity or Lipschitzness of gradients, by replacing the non-negativity threshold with interval constraints; the paper does not explore these extensions.
  • The exact check could be made into a certificate for small architectures: since the condition is a finite set of polynomial inequalities in the weights, symbolic verification of convexity is plausible for networks with few neurons.
  • The gap between ICNNs and general convex ReLU networks suggests that the common practice of enforcing convexity via ICNN architectures is a strong inductive bias, one that may exclude exactly the compact convex functions that wider, deeper unconstrained ReLU networks can interpolate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies convexity of ReLU neural networks and asks whether Input Convex Neural Networks (ICNNs) exhaust the set of convex functions representable by a given ReLU architecture. It provides a minimal characterization of convexity for continuous piecewise linear functions (Proposition 3.7), proves that for one-hidden-layer networks convexity forces the ICNN non-negative-weight condition (Proposition 4.9), and constructs a two-hidden-layer, two-neuron-per-layer network that implements a convex function not representable by any same-width ICNN (Proposition 2.1). For general DAG ReLU networks, the paper gives a necessary condition based on path-lifting (Proposition 6.3) and, under Assumption 7.1, an if-and-only-if characterization (Theorem 7.4). It also presents a numerical procedure, built on Berzins' polyhedral complex extraction, claimed to be an exact convexity check (Section 8 and Algorithm 1).

Significance. If the main characterization holds, the paper makes a substantial contribution: it shows that ICNNs are not the only way to obtain convex ReLU networks, gives a concrete and checkable counterexample, and opens a path toward architecture-aware convexity constraints. The proofs of the one-hidden-layer result and of the necessary conditions are detailed and appear sound, and the use of the path-lifting framework is elegant and well motivated. The counterexample of Proposition 2.1 is convincing and is a useful addition to the expressivity literature. However, the central sufficiency theorem and the claimed exactness of the numerical check rest on Assumption 7.1, whose genericity is only argued heuristically; this limits the strength of the paper's headline claims until the assumption is either proved to be generic or explicitly verified by the algorithm.

major comments (3)
  1. [Section 7.1 and 7.3, Theorem 7.4] The sufficiency direction of Theorem 7.4 depends on Assumption 7.1, specifically on the inclusion Fno-diff ⊆ ∪ν Xν in Eq. (30). The genericity discussion in Section 7.3 says the authors 'believe' a variant of Assumption 7.1 would follow from transversality and supertransversality, but no proof is supplied. When (30) fails, condition (31) is only necessary (Proposition 6.3), not sufficient, so the 'if and only if' statement is not established for all networks. This is load-bearing for the abstract's claim of 'necessary and sufficient conditions' for convex ReLU networks. The authors should either prove a genericity result or explicitly state Theorem 7.4 as conditional on an unverified assumption and moderate the abstract accordingly.
  2. [Section 8 and Appendix H, Algorithm 1] Algorithm 1 is presented as an 'exact check' of convexity, but it does not verify Assumption 7.1 before declaring a network convex. The perturbation step in Algorithm 1 line 6 takes sign-vectors of 1-faces, which have d−1 zeros, and replaces zeros with ±1 to obtain sign-vectors of d−1 faces. This presupposes that every d−1 face has exactly one zero, which is precisely condition (30). On a degenerate network satisfying the necessary condition (31) but violating (30), the algorithm can return True for a nonconvex function. The exactness claim in the abstract and Section 8 is therefore valid only under an unverified assumption. The algorithm should either verify Assumption 7.1 (or a substitute) or be described as a necessary-condition check rather than an exact convexity check.
  3. [Section 8 and Remark 8.1] The numerical convexity check is performed on a compact convex domain Ω, and Remark 8.1 asserts that the framework adapts by replacing Xν and F by their intersection with Ω. However, the proof of Theorem 7.4 relies on the global local characterisation Proposition 3.7(v), and no proof is given that checking the conditions only on Ω suffices for convexity on Ω. Since the algorithm is intended to certify convexity on a domain, the domain-restricted sufficiency needs a precise statement and proof; otherwise the experimental conclusions about 'convex ReLU networks' on the sampled domain are not fully justified.
minor comments (5)
  1. [Section 6.3 and Appendix F, Remark F.1] Theorem 7.4 states that the characterization 'holds even with max-pooling neurons', but Remark F.1 says that whether the proof for the ReLU case can be adapted to the max-pooling framework is left to future work. Since Lemma F.4 appears to provide the missing extension, this inconsistency should be resolved in the text.
  2. [Section 8.1, Equation (35)] The text compares the frequency of convex ReLU networks with the analytic ICNN probability, but the convex counts in Figure 3 include ICNNs. The statement that the frequency is '4 times greater' should be clarified to indicate that this is the ratio of all convex networks to ICNNs, not the ratio of non-ICNN convex networks to ICNNs.
  3. [Appendix G, Figure G4] The proof of Proposition 2.1 uses points A, B, and C in Figure G4, but the figure lacks axis labels and the coordinates of these points are not given in the caption; adding them would substantially improve readability of the case analysis.
  4. [Abstract] The abstract contains a grammatical error: 'The successes of deep learning has led' should be 'The successes of deep learning have led'.
  5. [Algorithm 1, Appendix H] In the pseudocode, the variable lν is used on line 10 but is not defined before; it should be introduced explicitly as the layer index of neuron ν.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the convexity characterizations are derived from first principles of convex CPWL analysis, and the counterexample is constructed from the characterization rather than used to fit it.

full rationale

The paper's central claims are self-contained derivations rather than re-statements of their inputs. Proposition 3.7 establishes minimal convexity conditions for CPWL functions directly from monotonicity of gradients and convexity along segments, with a proof in Appendix D. The necessary conditions for ReLU networks (Proposition 6.3) follow by applying Lemma 6.8, whose proof explicitly computes the slope difference and reduces the monotonicity inequality to the non-negativity of the path scalar product. The sufficiency direction (Theorem 7.4) uses Proposition 3.7(v) together with Assumption 7.1 to route every slope-changing frontier point through a single isolated neuron; this is a genuine argument, not a definitional equivalence. Proposition 2.1's counterexample is not fitted: it is verified by checking the derived inequalities (Section 5.2 and Appendix G), and the non-implementability by an ICNN with the same architecture is proved by exhaustive case analysis of possible ICNN parameterizations. The numerical procedure in Section 8 builds on the external Berzins algorithm for polyhedral complex extraction, which is independent of the paper's convexity conditions. The self-citations to Gonon et al. [32] and Stock and Gribonval [31] supply the path-lifting formalism and twin-neuron merging facts; these are tools, and the paper's convexity claims do not reduce to them. The main caveat is Assumption 7.1: the paper states in Section 7.3 that a variant of it would likely follow from transversality/supertransversality, but does not prove it, so Theorem 7.4's sufficiency and Algorithm 1's exactness are conditional on an unverified assumption. That is a correctness or assumption gap, not circularity: the assumption is not defined in terms of the conclusion, and the necessity direction remains valid regardless. No parameter is fitted and renamed as a prediction, no self-citation is load-bearing, and no known result is merely renamed. The honest finding is therefore no significant circularity, with the caveat about Assumption 7.1 noted as a non-circular limitation.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The central claim rests on standard convex analysis, the cited path-lifting framework, and two explicit non-degeneracy assumptions (4.4 and 7.1). No free parameters are fitted to data, and no new entities are postulated. The main burden is the sufficiency assumption 7.1, which is asserted generic but not fully proven.

assumptions (6)
  • standard math The path-lifting representation fθ(x) = ⟨Φ(θ), A(x,θ)(x;1)⟩ for DAG ReLU networks (Gonon et al.).
    Invoked in Section 6.1, Eq. (18), as a tool to express the network output; taken from prior work, not derived here.
  • standard math Convexity of CPWL functions can be checked via monotonicity of gradients at frontiers (Proposition 3.7).
    Core convex analysis fact, proved in Appendix D using subdifferential calculus and convexity along lines.
  • domain assumption Assumption 4.4: first-layer rows are non-zero and no two augmented rows are colinear; networks violating it can be reduced to smaller equivalent networks.
    Used to prove the one-hidden-layer equivalence; argued non-restrictive via neuron removal and twin-neuron merging in Proposition 4.9.
  • domain assumption Assumption 7.1: every hidden neuron is isolated and every slope-changing frontier point lies in some Xν.
    Needed for sufficiency in Theorem 7.4; the authors argue genericity in Section 7.3 but do not prove it fully. The numerical checker in Section 8 relies on this without verifying it.
  • domain assumption Berzins' algorithm exactly extracts all faces of the polyhedral complex of a ReLU network on a compact domain.
    Used in Section 8 to enumerate sign-vectors and activation patterns; exactness of the numerical check depends on this external algorithm.
  • standard math Standard convex-analysis facts: subdifferential characterization, convexity along lines, monotonicity of gradients, and composition rules for non-decreasing convex functions.
    Background for all proofs, listed in Fact C.5 through C.9 and Section 2.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Convexity in ReLU Neural Networks: beyond ICNNs?." pith.science (2026). https://pith.science/paper/3EA4NGPR

@misc{pith2026250103017,
  author       = {Pith},
  title        = {Pith review of: Convexity in ReLU Neural Networks: beyond ICNNs?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3EA4NGPR}},
  note         = {Machine review of arXiv:2501.03017}
}
read the original abstract

Convex functions and their gradients play a critical role in mathematical imaging, from proximal optimization to Optimal Transport. The successes of deep learning has led many to use learning-based methods, where fixed functions or operators are replaced by learned neural networks. Regardless of their empirical superiority, establishing rigorous guarantees for these methods often requires to impose structural constraints on neural architectures, in particular convexity. The most popular way to do so is to use so-called Input Convex Neural Networks (ICNNs). In order to explore the expressivity of ICNNs, we provide necessary and sufficient conditions for a ReLU neural network to be convex. Such characterizations are based on product of weights and activations, and write nicely for any architecture in the path-lifting framework. As particular applications, we study our characterizations in depth for 1 and 2-hidden-layer neural networks: we show that every convex function implemented by a 1-hidden-layer ReLU network can be also expressed by an ICNN with the same architecture; however this property no longer holds with more layers. Finally, we provide a numerical procedure that allows an exact check of convexity for ReLU neural networks with a large number of affine regions.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 40 canonical work pages

  1. [1]

    In: IEEE Global Conference on Signal and Information Processing, pp

    Venkatakrishnan, S., Bouman, C., Wohlberg, B.: Plug-and-play priors for model based reconstruction. In: IEEE Global Conference on Signal and Information Processing, pp. 945–948 (2013). IEEE

    work page 2013

  2. [2]

    In: Proceedings of the IEEE International Conference on Computer Vision, pp

    Meinhardt, T., Moller, M., Hazirbas, C., Cre- mers, D.: Learning proximal operators: Using denoising networks for regularizing inverse imaging problems. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1781–1790 (2017)

    work page 2017

  3. [3]

    IEEE transactions on image processing 26(7), 3142–3155 (2017)

    Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: Resid- ual learning of deep cnn for image denois- ing. IEEE transactions on image processing 26(7), 3142–3155 (2017)

    work page 2017

  4. [4]

    IEEE Transactions on Pattern Analysis and Machine Intelligence 44(10), 6360–6376 (2021)

    Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., Timofte, R.: Plug-and-play image restoration with deep denoiser prior. IEEE Transactions on Pattern Analysis and Machine Intelligence 44(10), 6360–6376 (2021)

    work page 2021

  5. [5]

    SIAM Journal on Imaging Sciences 14(3), 1206–1237 (2021)

    Pesquet, J.-C., Repetti, A., Terris, M., Wiaux, Y.: Learning maximally monotone operators for image recovery. SIAM Journal on Imaging Sciences 14(3), 1206–1237 (2021)

    work page 2021

  6. [6]

    Journal of Mathematical Imaging and Vision 62(6-7), 773–789 (2020)

    Gribonval, R., Nikolova, M.: A character- ization of proximity operators. Journal of Mathematical Imaging and Vision 62(6-7), 773–789 (2020)

    work page 2020

  7. [7]

    PhD the- sis, Universit´ e de Bordeaux (2023)

    Hurault, S.: Convergent plug-and-play meth- ods for image inverse problems with explicit and nonconvex deep regularization. PhD the- sis, Universit´ e de Bordeaux (2023)

    work page 2023

  8. [8]

    Cohen, R., Blau, Y., Freedman, D., Rivlin, E.: It has potential: Gradient-driven denois- ers for convergent solutions to inverse prob- lems (2021)

    work page 2021

Show all 42 references
  1. [9]

    In: ICLR (2024)

    Fang, Z., Buchanan, S., Sulam, J.: What’s in a prior? learned proximal networks for inverse problems. In: ICLR (2024)

    2024

  2. [10]

    Communications on pure and applied mathematics 44(4), 375–417 (1991)

    Brenier, Y.: Polar factorization and mono- tone rearrangement of vector-valued func- tions. Communications on pure and applied mathematics 44(4), 375–417 (1991)

    1991

  3. [11]

    In: ICLR (2021)

    Korotin, A., Egiazarian, V., Asadulaev, A., Safin, A., Burnaev, E.: Wasserstein-2 genera- tive networks. In: ICLR (2021)

    2021

  4. [12]

    In: ICML (2020)

    Makkuva, A., Taghvaei, A., Oh, S., Lee, J.: Optimal transport mapping via input convex neural networks. In: ICML (2020)

    2020

  5. [13]

    In: Neurips (2021)

    Korotin, A., Li, L., Genevay, A., Solomon, J.M., Filippov, A., Burnaev, E.: Do neural optimal transport solvers work? a continuous wasserstein-2 benchmark. In: Neurips (2021)

    2021

  6. [14]

    In: ICLR (2023)

    Amos, B.: On amortizing convex conjugates for optimal transport. In: ICLR (2023)

    2023

  7. [15]

    arXiv preprint arXiv:1910.12744 (2019)

    Saremi, S.: On approximating ∇f with neural networks. arXiv preprint arXiv:1910.12744 (2019)

    2019 arXiv

  8. [16]

    In: ICASSP 2023 (2023)

    Chaudhari, S., Pranav, S., Moura, J.M.: Learning gradients of convex functions with monotone gradient networks. In: ICASSP 2023 (2023). IEEE

    2023

  9. [17]

    In: ICML (2017)

    Amos, B., Xu, L., Kolter, J.Z.: Input convex neural networks. In: ICML (2017)

    2017

  10. [18]

    In: ICLR (2019)

    Chen, Y., Shi, Y., Zhang, B.: Optimal Control Via Neural Networks: A Convex Approach. In: ICLR (2019)

    2019

  11. [19]

    In: ICLR (2023)

    Korotin, A., Selikhanovych, D., Burnaev, E.: Neural optimal transport. In: ICLR (2023)

    2023

  12. [20]

    In: ICML (2024)

    Vesseron, N., Cuturi, M.: On a Neural Imple- mentation of Brenier’s Polar Factorization. In: ICML (2024)

    2024

  13. [21]

    ESAIM: Control, Optimisation and Calculus of Variations 22(3), 728–742 (2016)

    Bauschke, H.H., Lucet, Y., Phan, H.M.: On the convexity of piecewise-defined functions. ESAIM: Control, Optimisation and Calculus of Variations 22(3), 728–742 (2016)

    2016

  14. [22]

    unquantized relu neural networks 18 and beyond

    Gonon, A., Brisebarre, N., Gribonval, R., Riccietti, E.: Approximation speed of quan- tized vs. unquantized relu neural networks 18 and beyond. IEEE Transactions on Informa- tion Theory (2023)

    2023

  15. [23]

    Arora, R., Basu, A., Mianjy, P., Mukher- jee, A.: Understanding Deep Neural Networks with Rectified Linear Units. (2018)

    2018

  16. [24]

    Cambridge University Press, Cam- bridge (2004)

    Boyd, S., Vandenberghe, L.: Convex Opti- mization. Cambridge University Press, Cam- bridge (2004)

    2004

  17. [25]

    Journal of Computational and Applied Mathematics 441, 115667 (2024)

    Goujon, A., Etemadi, A., Unser, M.: On the number of regions of piecewise linear neu- ral networks. Journal of Computational and Applied Mathematics 441, 115667 (2024)

    2024

  18. [26]

    SIAM Journal on Applied Algebra and Geometry 6(2), 216–242 (2022)

    Grigsby, J.E., Lindsey, K.: On transversal- ity of bent hyperplane arrangements and the topological expressiveness of relu neural net- works. SIAM Journal on Applied Algebra and Geometry 6(2), 216–242 (2022)

    2022

  19. [27]

    arXiv preprint arXiv:2207.07696 (2022)

    Masden, M.: Algorithmic determination of the combinatorial structure of the lin- ear regions of relu neural networks. arXiv preprint arXiv:2207.07696 (2022)

    2022 arXiv

  20. [28]

    Optimization 31(3), 209–221 (1994)

    Gorokhovik, V.V., Zorko, O.I., Birkhoff, G.: Piecewise affine functions and polyhedral sets. Optimization 31(3), 209–221 (1994)

    1994

  21. [29]

    Bauschke, H., Combettes, P.: Convex Analy- sis and Monotone Operator Theory in Hilbert Spaces vol. 408. Springer, (2011)

    2011

  22. [30]

    In: Neurips (2019)

    Hanin, B., Rolnick, D.: Deep relu networks have surprisingly few activation patterns. In: Neurips (2019)

    2019

  23. [31]

    Constructive Approximation (2022)

    Stock, P., Gribonval, R.: An Embedding of ReLU Networks and an Analysis of their Identifiability. Constructive Approximation (2022). Publisher: Springer Verlag

    2022

  24. [32]

    In: ICLR (2023)

    Gonon, A., Brisebarre, N., Riccietti, E., Gri- bonval, R.: A path-norm toolkit for modern networks: consequences, promises and chal- lenges. In: ICLR (2023)

    2023

  25. [33]

    : An introduction to hyperplane arrangements

    Stanley, R.P., et al. : An introduction to hyperplane arrangements. Geometric combi- natorics 13(389-496), 24 (2004)

    2004

  26. [34]

    In: ICML (2020)

    Rolnick, D., Kording, K.: Reverse- engineering deep relu networks. In: ICML (2020)

    2020

  27. [35]

    In: CVPR (2023)

    Humayun, A.I., Balestriero, R., Balakrish- nan, G., Baraniuk, R.G.: Splinecam: Exact visualization and characterization of deep network geometry and decision boundaries. In: CVPR (2023)

    2023

  28. [36]

    In: ICML (2023)

    Berzins, A.: Polyhedral complex extraction from relu networks using edge subdivision. In: ICML (2023)

    2023

  29. [37]

    Springer, New York (1995)

    Ziegler, G.M.: Lectures on Polytopes. Springer, New York (1995)

    1995

  30. [38]

    Rockafellar, R.T.: Convex analysis (2015)

    2015

  31. [39]

    Comptes rendus hebdomadaires des s´ eances de l’Acad´ emie des sciences 257, 4117–4119 (1963)

    Moreau, J.J.: Fonctionnelles sous- diff´ erentiables. Comptes rendus hebdomadaires des s´ eances de l’Acad´ emie des sciences 257, 4117–4119 (1963)

    1963

  32. [40]

    k-max-pooling

    Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis vol. 317. Springer, Berlin (2009) 19 Appendix A Notations We list below recurrent symbols in this paper along with their description. n tuple of widths of hidden layers ICNNd(n) set of convex functions Rd → R implemented b...

    2009

  33. [41]

    there are x+, x− ∈ B(x, ϵ) such that fθ is differentiable at x+, x− and aν(x+, θ) = a+, aν(x−, θ) = a−

  34. [42]

    (F53) Proof

    for any such pair x+, x− it holds ⟨∇fθ(x+) − ∇fθ(x−), x+ − x−⟩ ≥0 ⇐ ⇒ ⟨aν→, Φν→(θ)⟩ ≥0 . (F53) Proof. Consider ν ∈ H an isolated neuron. In case of a ReLU neuron, aν(·, θ) = aν(·, θ), and without loss of generality a+ = 1, a−, so the result is a consequence of Lemma 6.8. Assum...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.