Pith. sign in

REVIEW 3 major objections 5 minor 46 references

Advancing Constrained Monotonic Neural Networks: Achieving Universal Approximation Beyond Bounded Activations

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A monotone MLP with non-negative weights and activations that saturate on alternating sides can interpolate any monotone function on any finite point set with three hidden layers, and flipping the weights to non-positive makes four-layer…

desk verdict Genuine extension of the four-layer monotone universal approximation result to all one-sided saturating activations, with a clean non-positive-ReLU corollary; the printed proof has a sign slip, a fixable tie gap, and stops at finite-point interpolation. read the letter →

arxiv 2505.02537 v2 pith:2E2VDAKT submitted 2025-05-05 cs.LG cs.AIstat.ML

Davide Sartor , Alberto Sinigaglia , Gian Antonio Susto This is my paper · ORCID
classification cs.LGcs.AIstat.ML MSC 68T0741A30
keywords monotonicneuralnetworksuniversalapproximationweightconstraintssaturatingactivationsReLUmonotoneinterpolationnon-positiveweightsactivationswitch
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Monotonic neural networks are models whose predictions are forced by construction to rise (or fall) with their inputs, and most architectures build this in with bounded activations like sigmoid, which are slow to train and cannot extrapolate beyond the data range. This paper argues that boundedness is not what gives these networks their universal approximation power; what matters is that each activation saturates on one side and consecutive layers saturate on alternating sides. The main theorem proves by explicit construction that a non-negative-weight MLP with three hidden layers interpolates any monotone function on any finite set of points under this alternation, matching the best depth bound previously known only for step-function activations. A second result shows that flipping the weight constraint to non-positive makes four-layer ReLU networks universal approximators for monotone functions, something the non-negative setting provably cannot do with convex activations. An 'activation switch' layer derived from this insight removes the need for weight reparameterization, and the paper's experiments show it matches or outperforms existing monotonic architectures.

What carries the argument

The load-bearing device is the saturation side of a monotone activation: a function is right-saturating (in $S^+$) when it has a finite limit at $+\infty$ and left-saturating (in $S^-$) when it has a finite limit at $-\infty$; ReLU, ELU, and softplus are left-saturating, while sigmoid and tanh saturate both sides. The proof mechanism is a three-stage geometric construction in which each hidden layer refines a set-valued approximation: first-layer units, using weight vectors $\alpha \geq 0$ scaled toward infinity, approximate indicator functions of half-spaces; second-layer units, whose activation saturates in the opposite direction, combine those indicators into indicator functions of intersections (the sets $A_i^{(2)}$ built from the half-spaces that contain $x_i$ and exclude later points); third-layer units, saturating back in the original direction, form the indicator of the superlevel set via complements of intersections; and the fourth layer's non-negative weights $w_j = (f(x_j) - f(x_{j-1}))/\gamma^{(3)}$ telescope into an exact interpolation. The other load-bearing identity is the sign-flip equivalence, Proposition 3.10: two adjacent layers with non-positive weights and activation $\sigma$ compute exactly what two non-negative layers would compute with the point-reflected activation $\sigma'(x) = -\sigma(-x)$, and repeated application of this flip converts the alternating-saturation theorem into the non-positive-weight universal approximation result.

What would settle it

Take the constant function $f \equiv 0$ on the two points $x_1 = (1,1)$ and $x_2 = (0,0)$ in $\mathbb{R}^2$, ordered $x_1$ before $x_2$ as the theorem's 'ties resolved arbitrarily' permits. For every non-negative vector $\alpha$, we have $\alpha \cdot x_2 = 0 \leq \alpha \cdot x_1$, so no hyperplane with non-negative normal can put $x_2$ strictly on its positive side and $x_1$ strictly on its negative side; the Layer-1 separation the proof invokes for every pair therefore cannot be instantiated on this input, showing the proof's claim that monotonicity alone guarantees such a separating hyperplane is false as stated.

Watch

Extended reading notes

Core claim

The paper's central claim is that the saturation side of a monotone activation, not boundedness, confers universal approximation power on weight-constrained MLPs. Theorem 3.5 states that an MLP with non-negative weights and three hidden layers can interpolate any monotone non-decreasing function on any finite set of points, provided the activations are monotone and alternate saturation sides: $\sigma^{(1)} \in S^-$, $\sigma^{(2)} \in S^+$, $\sigma^{(3)} \in S^-$, or the reverse, where $S^-$ and $S^+$ denote activations with finite limits at $-\infty$ and $+\infty$. The construction builds indicator functions of half-spaces in the first layer, of intersections in the second, of the superlevel sets $\{x_j : f(x_j) \geq f(x_i)\}$ in the third, and finishes with a fourth layer whose non-negative weights telescope the differences between consecutive function values, so the output equals $f(x_i)$ at every training point. Because this bound does not depend on input dimension, it matches the best-known constant-layer result, previously available only for threshold activations. The companion result, Proposition 3.11, converts the alternation theorem into a statement about the sign of the weights: with four layers and non-positive weights, any saturating monotone activation, plain ReLU included, yields a universal approximator for monotone functions, in contrast to the convex-function-only limitation of non-negative ReLU networks. The paper also packages this into a practical layer form, the post-activation switch $\hat{f}(x) = W^+ \sigma(x) + W^- \sigma(-x) + b$, which lets the network learn its activation's saturation side rather than fixing it in advance.

Load-bearing premise

The construction in the proof of Theorem 3.5 assumes that every pair of training points with $i < j$, including pairs with equal function values, can be separated by a hyperplane with non-negative normal so that $x_j$ falls on the positive side; this can fail when the points are comparable in the componentwise order but their function values tie, and the proof's 'ties resolved arbitrarily' does not ensure a separating orientation.

Editorial extensions

If this is right

  • Any one-sided saturating monotone activation, including ReLU, ELU, CELU, SELU, softplus, exponential, tanh, or sigmoid, can replace bounded activations in monotonic MLPs while preserving universal approximation at constant four-layer depth.
  • Non-positive weight constraints make four-layer ReLU networks universal approximators for monotone functions, whereas the non-negative counterpart with ReLU can only approximate convex functions.
  • The activation-switch layer, $\hat{f}(x) = W^+ \sigma(x) + W^- \sigma(-x) + b$, needs no weight reparameterization and no a priori choice of alternating activations, which the paper's experiments show eases initialization and improves training stability.
  • Existing constrained architectures that use three activation classes to obtain universal monotonic approximation need only an activation and its point reflection; the third, bounded class is redundant.
  • On the five benchmark datasets reported, the proposed layer matches or outperforms prior monotonic architectures without special weight initialization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper proves interpolation on finite point sets and asserts universal approximation on compact domains; the unstated bridge is a grid-density argument for continuous monotone functions, and spelling it out would make the abstract's claim fully rigorous.
  • The proof's phrase 'ties resolved arbitrarily' hides an assumption: equal-valued point pairs must still be separable by a hyperplane with non-negative normal. Ordering ties so that no later point is componentwise smaller than an earlier equal-valued point would repair the construction, if such an ordering always exists.
  • Because the activation-switch layer learns its saturation direction per neuron, the alternating pattern could be dropped or learned end-to-end; a testable variant is per-neuron gating of the $W^+$/$W^-$ split, which would preserve the monotonicity guarantee only if the weights feeding the gate stay non-negative.
  • The paper leaves open whether non-saturating activations such as Leaky-ReLU can be used; a concrete next test is whether a minimally saturating modification, any finite limit on one side, inherits the theorem, which would settle the boundary of the result.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies MLPs with constrained weights for monotonic function approximation. It claims that (i) a non-negative-weight MLP with three hidden layers and monotone activations saturating on alternating sides can interpolate any monotone function on any finite point set; (ii) as a consequence, four-layer non-negative MLPs with alternating sigma and its point reflection, and four-layer non-positive MLPs with any one-sided saturating activation (including ReLU), are universal approximators for monotone functions; and (iii) an 'activation-switch' parametrization that uses the positive and negative parts of the weight matrix achieves the same expressivity without explicit weight constraints and trains stably. The proof constructs half-space indicators with large weights, forms level sets through intersections and complements, and sums a telescoping series in the output layer. Experiments compare the proposed parametrization with several monotonic architectures on five datasets.

Significance. If the theoretical claims were fully established, the paper would make a useful contribution: it extends the four-layer universal approximation result of Mikulincer and Reichman (2022) from threshold activations to all one-sided saturating monotone activations, and it gives the practically relevant result that non-positive weight constraints with ReLU avoid the convexity limitation of non-negative constraints. The proposed activation-switch parametrization is simple, removes the need to preselect alternating activations, and the reported empirical results are competitive. However, the advertised universal approximation is not proved as written: the paper establishes only finite-sample approximate interpolation, and one step in that proof is flawed. The gaps are repairable, but they are load-bearing for the central claims.

major comments (3)
  1. [Sections 3.2-3.3, Propositions 3.9 and 3.11] Theorem 3.5 proves only approximate interpolation on an arbitrary finite set of points; it does not by itself imply universal approximation on a compact domain. Propositions 3.9 and 3.11 assert universal approximation without supplying the usual finite-grid argument, so the paper's central advertised claim is not established as written. The missing step is load-bearing: for a compact K and tolerance epsilon, one must choose a finite net whose cells have small f-variation (using uniform continuity of f), apply the finite construction to the values of f on the net, and then use monotonicity of both f and the constructed network to bound the sup-norm error in each cell. Please add this argument and explicitly state the target class and topology, e.g., continuous monotone functions on compact subsets of R^d with the sup norm.
  2. [Theorem 3.5, proof of Layer 1] The proof assumes that for every pair with i<j there is a non-negative normal alpha such that x_j is in the positive half-space and x_i is in the negative half-space. This is false when f(x_i)=f(x_j) and x_j is componentwise at most x_i, because every alpha>=0 then satisfies alpha^T x_j <= alpha^T x_i. The phrase 'ties resolved arbitrarily' does not prevent this situation. The gap is repairable by ordering equal-valued points so that componentwise-smaller points come first, but as printed the proof is invalid at this step.
  3. [Theorem 3.5 and Lemmas 3.6-3.7] The theorem states that the network can 'interpolate' the target function, but the proof uses approximations throughout: Lemma 3.6 and Lemma 3.7 only show that indicators are approximated for large lambda, and no limit is taken. With one-sided saturating activations such as ReLU, exact step functions are not representable, so the exact-interpolation statement is stronger than what is proved. Please either weaken the theorem to epsilon-approximate interpolation with an explicit error bound, or give an exactness argument for the activations for which exact interpolation holds.
minor comments (5)
  1. [Lemma 3.6 and Lemma A.3] In the proof of Lemma 3.6 (and its appendix counterpart), the parameters w=lambda alpha^T and b=lambda alpha^T beta give sigma(lambda alpha^T(x+beta)), not sigma(lambda alpha^T(x-beta)); the correct bias is b=-lambda alpha^T beta. Since beta is arbitrary, this is a local algebraic slip that does not affect the existence of the desired half-space indicators, but it should be corrected.
  2. [Theorem 3.5] Theorem 3.5 says '3 hidden layers' while the proof uses four affine layers (three hidden layers plus an output layer). The layer count should be stated consistently with Propositions 3.9 and 3.11, which refer to 'at least 4 layers'.
  3. [Lemma 3.7] The intersection in Lemma 3.7 is written as A=intersection_{i=0}^n A_i, but the following text refers to A_1,...,A_n; the index should run from 1 to n.
  4. [Proposition 3.10] Proposition 3.10 is stated without proof; since it is the basis for Proposition 3.11, a short derivation showing how the bias terms transform under the double sign flip would make the equivalence easier to verify.
  5. [Section 5, Table 1] The experimental section does not report the number of random seeds or how the confidence intervals in Table 1 were computed; please add these details so the empirical comparison is reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the universal-approximation claims are derived from constructive lemmas and algebraic equivalences, not from fitting or self-citation.

full rationale

The paper's central derivation chain is self-contained. Theorem 3.5 is proved by an explicit layer-by-layer construction using Lemmas 3.6 and 3.7, and the final-layer weights are written in closed form as differences of the target function values, so the interpolation conclusion is not baked into a fitted parameter or a renamed input. Proposition 3.10 is a direct algebraic identity showing that two adjacent non-positive weight matrices with activation sigma are equivalent to two non-negative weight matrices with the point-reflected activation sigma-prime; Propositions 3.9 and 3.11 are corollaries of that identity together with Theorem 3.5, not restatements of the theorem's assumptions. The only self-citations in the reference list concern a dataset source and are not load-bearing for any theoretical claim. The paper does not invoke a uniqueness theorem from the authors' own prior work, and its main external dependence is on Mikulincer and Reichman and on Runje and Shankaranarayana, whose results are used as prior art rather than as a substitute for proof. The apparent weaknesses in the paper, such as the tie-ordering condition needed for the separating hyperplanes in Theorem 3.5 and the missing compactness argument that would turn finite-point interpolation into uniform approximation on compact domains, are mathematical gaps or overstatements rather than circularities: they do not fit the conclusion into the inputs and do not define the result in terms of itself. Therefore the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The theory introduces no fitted parameters and no invented entities. Its main external input is the geometric separation premise and the use of limits; these are standard but under-specified in the proof. The empirical hyperparameters are taken from prior work, not fitted to the central theoretical claim.

assumptions (3)
  • domain assumption For every pair of ordered points with strictly increasing f values, a separating hyperplane with non-negative normal exists.
    Used in Layer 1 of the proof of Theorem 3.5; follows from monotonicity only when f(x_i)<f(x_j), and fails for equal-valued componentwise comparable points unless tie order is chosen carefully.
  • standard math The limit constructions in Lemmas 3.6 and 3.7 approximate the desired indicators with arbitrarily small error on finite separated point sets.
    The proof uses 'approximately' and limits as λ→∞ without explicit epsilon-N quantification; standard for finite sets, but the paper does not formalize it.
  • standard math Composition of monotone functions preserves monotonicity (Remark 3.1).
    Basis for the claim that alternating activation signs do not break monotonicity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Advancing Constrained Monotonic Neural Networks: Achieving Universal Approximation Beyond Bounded Activations." pith.science (2026). https://pith.science/paper/2E2VDAKT

@misc{pith2026250502537,
  author       = {Pith},
  title        = {Pith review of: Advancing Constrained Monotonic Neural Networks: Achieving Universal Approximation Beyond Bounded Activations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2E2VDAKT}},
  note         = {Machine review of arXiv:2505.02537}
}
read the original abstract

Conventional techniques for imposing monotonicity in MLPs by construction involve the use of non-negative weight constraints and bounded activation functions, which pose well-known optimization challenges. In this work, we generalize previous theoretical results, showing that MLPs with non-negative weight constraint and activations that saturate on alternating sides are universal approximators for monotonic functions. Additionally, we show an equivalence between the saturation side in the activations and the sign of the weight constraint. This connection allows us to prove that MLPs with convex monotone activations and non-positive constrained weights also qualify as universal approximators, in contrast to their non-negative constrained counterparts. Our results provide theoretical grounding to the empirical effectiveness observed in previous works while leading to possible architectural simplification. Moreover, to further alleviate the optimization difficulties, we propose an alternative formulation that allows the network to adjust its activations according to the sign of the weights. This eliminates the requirement for weight reparameterization, easing initialization and improving training stability. Experimental evaluation reinforces the validity of the theoretical results, showing that our novel approach compares favourably to traditional monotonic architectures.

Figures

Figures reproduced from arXiv: 2505.02537 by the authors.

Figure 1
Figure 1. Monotone MLPs with weight-constraint and bounded activations (pink) and our proposed approach based on ReLU (blue). The former can only represent bounded functions and, thus, cannot extrapolate the data trend, which is important in many domains, such as time-series analysis and predictive maintenance. 1. Introduction Monotonic neural networks represent a pivotal shift in deep learning. They bridge the gap between hi… view at source ↗
Figure 2
Figure 2. Constructions of Heavyside function using a com￾position of ReLU and its point reflection ReLU’ to obtain ReLU(ReLU′ (αx − 0.5) + 1) = ReLU′ (ReLU(αx + 0.5) − 1) Definition 3.3. Given a function σ : R → R saturates right/left if the corresponding limit exists and is finite. That is, σ is right-saturating if σ(+∞) := limx→+∞ σ(x) ∈ R, and it is left-saturating if σ(−∞) := limx→−∞ σ(x) ∈ R. We will denote the set of r… view at source ↗
Figure 3
Figure 3. Example of representable functions at layer 1. Using Lemma 3.6, we can ensure that it is possible to have: ( h (1) i (x) ≈ σ (1)(−∞) = 0, if x ∈ A − j/i ∀i, j h (1) i (x) ≈ σ (1)(+∞) > 0, otherwise (6) A visual example is shown in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Example of representable functions at layer 2. Layer 3 Consider A (3) i = T j:j<i A¯(2) j , where A¯(2) j is the complement of A (2) j . Using Equation (7) we can once again 3 In this case γ (2) < 0 since by assumption σ (2) saturates right. 5 [PITH_FULL_IMAGE:figures…
Figure 5
Figure 5. Figure 5: Example of representable functions at layer 3. Layer 4 To conclude the proof, simply take the weights at the fourth layer to be : w =  f(x1) − b γ (3) , f(x2) − f(x1) γ (3) , . . . , f(xn) − f(xn−1) γ (3)  Since the points are ordered, this ensures that w contains al…
Figure 6
Figure 6. Figure 6: Computation graph of a single layer of a monotonic NN with the proposed learned activation via weight sign. further hyperparameter tuning. However, by slightly rear￾ranging the order of operations, it is possible to construct a monotone MLP that does not require manual…
Figure 7
Figure 7. Figure 7: First plot, the distribution of the output of an MLP with the different parametrizations. Second plot, the scaling law of the expected output after initialization of the different parametrization. In both images, it can be seen how the naive constrained MLP has a very …
Figure 8
Figure 8. Figure 8: First plot, approximation of f(x) using MLPs with layers of 128 neurons. Second plot, approximation of f(x) using MLPs with layers of 256 neurons. Last plot, training losses of the different methods (full lines represent versions with 128 neurons, dashed lines represen…
Figure 9
Figure 9. Figure 9: Average gradient from monotonic MLPs varying the number of layers. Data is shown in the log scale for the y-axis. 27.5 25.0 22.5 20.0 17.5 15.0 12.5 10.0 7.5 log(|df (x) d i |) 0.0 0.2 0.4 0.6 0.8 1.0 Sigmoid MLP layer 0 layer 1 layer 2 layer 3 layer 4 layer 5 layer 6 …
Figure 10
Figure 10. Figure 10: Distribution of gradients from monotonic MLPs for each layer (layer 0 is the final one, layer 6 is the first after the input). 16 [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Computation graph of a single layer of a ReLU monotonic NN with the proposed learned activation via weight sign. The left plot reports the computational graph of the post-activation, and the right plot shows the pre-activation switch. Algorithm 2 Forward pass of a Mon…
Figure 12
Figure 12. Figure 12: Examples of learnable functions at the first hidden layer. 7Note that σ (1)(±∞) needs not be finite. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]
Figure 13
Figure 13. Figure 13: Examples of learnable indicator functions at the second hidden layer. Third layer construction Finally, let us show that a hidden unit in the third layer can perform union and intersection operations when the second-layer representations are indicator functions of set…
Figure 14
Figure 14. Figure 14: Examples of learnable functions at the third hidden layer. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 35 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Machine bias

    Angwin, J., Larson, J., Mattu, S., and Kirchner, L. Machine bias. In Ethics of data and analytics, pp.\ 254--264. Auerbach Publications, 2022

    work page 2022

  3. [3]

    Barron, J. T. Continuously differentiable exponential linear units. arXiv preprint arXiv:1704.07483, 2017

    arXiv 2017

  4. [4]

    Feedback prediction for blogs

    Buza, K. Feedback prediction for blogs. In Data analysis, machine learning and knowledge discovery, pp.\ 145--152. Springer, 2013

    work page 2013

  5. [5]

    and Guestrin, C

    Chen, T. and Guestrin, C. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp.\ 785--794, 2016

    2016

  6. [6]

    and Silva, R

    Chilinski, P. and Silva, R. Neural likelihoods via cumulative distribution functions. In Conference on Uncertainty in Artificial Intelligence, pp.\ 420--429. PMLR, 2020

    work page 2020

  7. [7]

    Fast and accurate deep network learning by exponential linear units (elus)

    Clevert, D.-A., Unterthiner, T., and Hochreiter, S. Fast and accurate deep network learning by exponential linear units (elus). In ICLR, 2016

    work page 2016

  8. [8]

    and Velikova, M

    Daniels, H. and Velikova, M. Monotone and partially monotone neural networks. IEEE Transactions on Neural Networks, 21 0 (6): 0 906--917, 2010

    work page 2010

Show all 46 references
  1. [9]

    and Farid, H

    Dressel, J. and Farid, H. The accuracy, fairness, and limits of predicting recidivism. Science advances, 4 0 (1): 0 eaao5580, 2018

    2018

  2. [10]

    R., Singh, S

    Dubey, S. R., Singh, S. K., and Chaudhuri, B. B. Activation functions in deep learning: A comprehensive survey and benchmark. Neurocomputing, 503: 0 92--108, 2022

    2022

  3. [11]

    Incorporating second-order functional knowledge for better option pricing

    Dugas, C., Bengio, Y., B \'e lisle, F., Nadeau, C., and Garcia, R. Incorporating second-order functional knowledge for better option pricing. Advances in neural information processing systems, 13, 2000

    2000

  4. [12]

    Incorporating functional knowledge in neural networks

    Dugas, C., Bengio, Y., B \'e lisle, F., Nadeau, C., and Garcia, R. Incorporating functional knowledge in neural networks. Journal of Machine Learning Research, 10 0 (6), 2009

    2009

  5. [13]

    Fabris, A., Messina, S., Silvello, G., and Susto, G. A. Algorithmic fairness datasets: the story so far. Data Mining and Knowledge Discovery, 36 0 (6): 0 2074--2152, 2022

    2022

  6. [14]

    and Bengio, Y

    Glorot, X. and Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp.\ 249--256. JMLR Workshop and Conference Proceedings, 2010

    2010

  7. [15]

    Maxout networks

    Goodfellow, I., Warde-Farley, D., Mirza, M., Courville, A., and Bengio, Y. Maxout networks. In International conference on machine learning, pp.\ 1319--1327. PMLR, 2013

    2013

  8. [16]

    How to incorporate monotonicity in deep networks while preserving flexibility? in NeurIPS, 2019

    Gupta, A., Shukla, N., Marla, L., Kolbeinsson, A., and Yellepeddi, K. How to incorporate monotonicity in deep networks while preserving flexibility? in NeurIPS, 2019

    2019

  9. [17]

    Monotonic calibrated interpolated look-up tables

    Gupta, M., Cotter, A., Pfeifer, J., Voevodski, K., Canini, K., Mangylov, A., Moczydlowski, W., and Van Esbroeck, A. Monotonic calibrated interpolated look-up tables. Journal of Machine Learning Research, 17 0 (109): 0 1--47, 2016

    2016

  10. [18]

    Deep residual learning for image recognition

    He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 770--778, 2016

    2016

  11. [19]

    Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem

    Hein, M., Andriushchenko, M., and Bitterwolf, J. Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 41--50, 2019

    2019

  12. [20]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift

    Ioffe, S. Batch normalization: Accelerating deep network training by reducing internal covariate shift. in ICML, 2015

    2015

  13. [21]

    H., Tom, B., and Barrett, J

    Jeanselme, V., Yoon, C. H., Tom, B., and Barrett, J. Neural fine-gray: Monotonic neural networks for competing risks. In Conference on Health, Inference, and Learning, pp.\ 379--392. PMLR, 2023

    2023

  14. [22]

    Deep learning with s-shaped rectified linear activation units

    Jin, X., Xu, C., Feng, J., Wei, Y., Xiong, J., and Yan, S. Deep learning with s-shaped rectified linear activation units. In Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016

    2016

  15. [23]

    and Lee, J.-S

    Kim, H. and Lee, J.-S. Scalable monotonic neural networks. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=DjIsNDEOYX

    2024

  16. [24]

    Self-normalizing neural networks

    Klambauer, G., Unterthiner, T., Mayr, A., and Hochreiter, S. Self-normalizing neural networks. Advances in neural information processing systems, 30, 2017

    2017

  17. [25]

    Certified monotonic neural networks

    Liu, X., Han, X., Zhang, N., and Liu, Q. Certified monotonic neural networks. Advances in Neural Information Processing Systems, 33: 0 15427--15438, 2020

    2020

  18. [26]

    and Reichman, D

    Mikulincer, D. and Reichman, D. Size and depth of monotone neural networks: interpolation and approximation. Advances in Neural Information Processing Systems, 35: 0 5522--5534, 2022

    2022

  19. [27]

    Fast and flexible monotonic functions with ensembles of lattices

    Milani Fard, M., Canini, K., Cotter, A., Pfeifer, J., and Gupta, M. Fast and flexible monotonic functions with ensembles of lattices. Advances in neural information processing systems, 29, 2016

    2016

  20. [28]

    Mitchell, T. M. The need for biases in learning generalizations. Technical Report CBM-TR-117, Carnegie Mellon University, 1980

    1980

  21. [29]

    and Hinton, G

    Nair, V. and Hinton, G. E. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10), pp.\ 807--814, 2010

    2010

  22. [30]

    and Mart \' nez, M

    Nguyen, A.-p. and Mart \' nez, M. R. Mononet: towards interpretable models by learning monotonic features. In NeurIPS, 2019

    2019

  23. [31]

    Expressive monotonic neural networks

    Nolte, N., Kitouni, O., and Williams, M. Expressive monotonic neural networks. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=w2P7fMy_RH

    2023

  24. [32]

    Fully neural network based model for general temporal point processes

    Omi, T., Aihara, K., et al. Fully neural network based model for general temporal point processes. Advances in neural information processing systems, 32, 2019

    2019

  25. [33]

    Pytorch: An imperative style, high-performance deep learning library

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019

    2019

  26. [34]

    On the expressive power of deep neural networks

    Raghu, M., Poole, B., Kleinberg, J., Ganguli, S., and Sohl-Dickstein, J. On the expressive power of deep neural networks. In international conference on machine learning, pp.\ 2847--2854. PMLR, 2017

    2017

  27. [35]

    and Sriraman, H

    Ravikumar, A. and Sriraman, H. Mitigating vanishing gradient in sgd optimization in neural networks. In International Conference on Information, Communication and Computing Technology, pp.\ 1--11. Springer, 2023

    2023

  28. [36]

    and Shankaranarayana, S

    Runje, D. and Shankaranarayana, S. M. Constrained monotonic neural networks. In International Conference on Machine Learning, pp.\ 29338--29353. PMLR, 2023

    2023

  29. [37]

    Simplified models of remaining useful life based on stochastic orderings

    S \'a nchez, L., Costa, N., and Couso, I. Simplified models of remaining useful life based on stochastic orderings. Reliability Engineering & System Safety, 237: 0 109321, 2023

    2023

  30. [38]

    and Abu-Mostafa, Y

    Sill, J. and Abu-Mostafa, Y. Monotonicity hints. Advances in neural information processing systems, 9, 1996

    1996

  31. [39]

    Counterexample-guided learning of monotonic neural networks

    Sivaraman, A., Farnadi, G., Millstein, T., and Van den Broeck, G. Counterexample-guided learning of monotonic neural networks. Advances in Neural Information Processing Systems, 33: 0 11936--11948, 2020

    2020

  32. [40]

    Review and comparison of commonly used activation functions for deep neural networks

    Szanda a, T. Review and comparison of commonly used activation functions for deep neural networks. Bio-inspired neurocomputing, pp.\ 203--224, 2021

    2021

  33. [41]

    and Lopez-Paz, D

    Tagasovska, N. and Lopez-Paz, D. Single-model uncertainties for deep learning. Advances in neural information processing systems, 32, 2019

    2019

  34. [42]

    Attention is all you need

    Vaswani, A. Attention is all you need. Advances in Neural Information Processing Systems, 2017

    2017

  35. [43]

    The resurgence of structure in deep neural networks

    Veli c kovi \'c , P. The resurgence of structure in deep neural networks. PhD thesis, University of Cambridge, 2019

    2019

  36. [44]

    and Louppe, G

    Wehenkel, A. and Louppe, G. Unconstrained monotonic neural networks. Advances in neural information processing systems, 32, 2019

    2019

  37. [45]

    Hierarchical lattice layer for partially monotone neural networks

    Yanagisawa, H., Miyaguchi, K., and Katsuki, T. Hierarchical lattice layer for partially monotone neural networks. Advances in Neural Information Processing Systems, 35: 0 11092--11103, 2022

    2022

  38. [46]

    Deep lattice networks and partial monotonic functions

    You, S., Ding, D., Canini, K., Pfeifer, J., and Gupta, M. Deep lattice networks and partial monotonic functions. Advances in neural information processing systems, 30, 2017

    2017

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.