pith. sign in

arxiv: 2510.23634 · v3 · pith:MS3LZUHNnew · submitted 2025-10-24 · 💻 cs.LG · cs.AI

Monotone and Separable Set Functions: Characterizations and Neural Models

Soutrik Sarangi , Yonatan Sverdlov , Nadav Dym , Abir De This is my paper

Pith reviewed 2026-05-21 20:01 UTC · model grok-4.3

classification 💻 cs.LG cs.AI
keywords monotone set functionsset embeddingsorder preservationneural networks for setsset containmentuniversal approximation
0
0 comments X

The pith

Set-to-vector functions preserve exact set containment if and only if the ground set is finite.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper defines monotone and separating set functions that map sets to vectors while preserving the subset relation exactly through componentwise vector inequality. It derives matching lower and upper bounds on the vector dimension needed, expressed in terms of the cardinality of the ground set and the multisets under consideration. For infinite ground sets such exact embeddings are proven impossible, but a neural architecture is supplied that satisfies a relaxed version of the property and remains stable. These embeddings further enable the construction of neural networks that are monotone by construction and can approximate every monotone set function to any desired accuracy.

Core claim

Monotone and separating (MAS) set functions are maps F from sets to vectors such that S is contained in T exactly when F(S) is componentwise less than or equal to F(T). The paper establishes lower and upper bounds on the dimension of the target vectors for finite ground sets and shows that no such functions exist for infinite ground sets; a neural model is instead given that achieves a weakly MAS property while being Hölder continuous. MAS functions are further shown to support universal models that are monotone by construction and dense among all monotone set functions.

What carries the argument

The MAS property, requiring that a set-to-vector map F satisfies S ⊆ T if and only if F(S) ≤ F(T) componentwise; this exact biconditional determines the dimension bounds and the non-existence result for infinite domains.

If this is right

  • For finite ground sets the minimal embedding dimension is bounded above and below by explicit expressions in the ground-set size and multiset cardinality.
  • MAS embeddings yield neural networks that are monotone by construction and can approximate any monotone set function.
  • A Hölder-continuous neural model satisfies a relaxed version of the MAS property when the ground set is infinite.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The lower bounds on dimension suggest that exact MAS embeddings become impractical for very large finite ground sets, motivating approximate relaxations in applications.
  • The benefit of the weakly MAS inductive bias could be quantified by comparing the neural model against standard set architectures on larger containment benchmarks.
  • Analogous order-preserving characterizations may extend to other partial orders arising in combinatorial optimization or relational learning.

Load-bearing premise

The embedding must satisfy the exact biconditional between set containment and vector inequality rather than an approximate or one-sided version.

What would settle it

An explicit MAS embedding for a chosen finite ground set and multiset size whose dimension lies strictly below the paper's lower bound, or a concrete pair of sets on which the proposed neural model violates the weakly MAS property.

Figures

Figures reproduced from arXiv: 2510.23634 by Abir De, Nadav Dym, Soutrik Sarangi, Yonatan Sverdlov.

Figure 2
Figure 2. Figure 2: Using a multiset model as in (2) with activations which are not always non-negative, like σ = Tanh will not be monotone. ReLU will be monotone, but to be weakly MAS, two layers are required. TRI and more general hat functions are weakly MAS even with a single layer. (A, b). If in addition σ is monotone (increasing or decreasing), then there exists S ̸⊆ T with F(S; (A, b)) ≤ F(T; (A, b)) for all A, b. Proof… view at source ↗
Figure 4
Figure 4. Figure 4: Acc vs |T| for 1-layer MLP for varying values of |S| and |T|. We observe that: (1) Both variants of our method outperform the baselines; (2) the base￾lines degrade significantly as |T| increases (with fixed |S|), as predicting separability becomes harder as the gap between |T| and |S| increases; (3) The performance of M-ReLU and M￾Hat is comparable. We now compare 1-layer pointwise mod￾els ending in ReLU a… view at source ↗
Figure 8
Figure 8. Figure 8: Acc vs |T| for all models ≥ 2 layer MLP [PITH_FULL_IMAGE:figures/full_fig_p031_8.png] view at source ↗
read the original abstract

Motivated by applications for set containment problems, we consider the following fundamental problem: can we design set-to-vector functions so that the natural partial order on sets is preserved, namely $S\subseteq T \text{ if and only if } F(S)\leq F(T) $. We call functions satisfying this property Monotone and Separating (MAS) set functions. % We establish lower and upper bounds for the vector dimension necessary to obtain MAS functions, as a function of the cardinality of the multisets and the underlying ground set. In the important case of an infinite ground set, we show that MAS functions do not exist, but provide a model called our which provably enjoys a relaxed MAS property we name "weakly MAS" and is stable in the sense of Holder continuity. We also show that MAS functions can be used to construct universal models that are monotone by construction and can approximate all monotone set functions. Experimentally, we consider a variety of set containment tasks. The experiments show the benefit of using our our model, in comparison with standard set models which do not incorporate set containment as an inductive bias. Our code is available in https://github.com/structlearning/MASNET.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 3 minor

Summary. The paper defines Monotone and Separable (MAS) set functions F: multisets → R^d satisfying the exact biconditional S ⊆ T iff F(S) ≤ F(T) under the product order. It derives lower and upper bounds on the minimal d as a function of maximum multiset cardinality and ground-set size. For infinite ground sets it proves non-existence of any finite-d MAS function and introduces a neural model (referred to as “our model”) that satisfies a relaxed “weakly MAS” property together with Hölder continuity. The paper further shows that MAS embeddings can be used to construct universal approximators that are monotone by construction. Experiments on set-containment tasks report gains over standard set neural networks; code is released.

Significance. If the dimension bounds and non-existence result are rigorously established, the work supplies fundamental limits on exact order-preserving set embeddings and a practical, stable workaround for infinite domains. The universal-approximation construction for monotone set functions and the inductive-bias experiments are directly useful for neural models on sets and graphs. Reproducibility via released code is a clear strength.

major comments (2)
  1. [§4, Theorem 2] §4, Theorem 2 (lower bound): the counting argument establishing the dimension lower bound appears to treat multisets as ordinary sets when enumerating distinct inclusions; if repeated elements are allowed, the number of distinct containment relations increases and the claimed lower bound may no longer be tight.
  2. [§5.2, Proposition 4] §5.2, Proposition 4 (non-existence): the proof invokes a finite-dimensional embedding and derives a contradiction via an infinite descending chain; the argument assumes the codomain is R^d with the standard product order, but does not explicitly rule out other finite-dimensional ordered vector spaces (e.g., with different cones), which is load-bearing for the claim that no MAS function exists at all.
minor comments (3)
  1. [Abstract] Abstract, line 8: repeated phrase “our our model” should be replaced by the model name.
  2. [§6.1] §6.1: the neural architecture realizing the weakly-MAS property would be clearer with an explicit pseudocode block or layer diagram.
  3. [§7] §7 (Experiments): while gains are reported, the main text should include dataset cardinalities, exact train/validation/test splits, and standard-error bars to allow direct replication of the containment-task results.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive comments on our manuscript. We address each major comment below and will make the indicated revisions to improve clarity.

read point-by-point responses

  1. Referee: [§4, Theorem 2] §4, Theorem 2 (lower bound): the counting argument establishing the dimension lower bound appears to treat multisets as ordinary sets when enumerating distinct inclusions; if repeated elements are allowed, the number of distinct containment relations increases and the claimed lower bound may no longer be tight.

    Authors: We appreciate the referee's observation regarding the treatment of multisets. In the proof of Theorem 2, we consider the poset of multisets of cardinality at most k over an n-element ground set, where containment is defined componentwise on multiplicity vectors. The counting argument lower-bounds the dimension by injecting a suitable collection of incomparable multisets (accounting for all possible non-negative integer multiplicity assignments) into the image under F. While repetitions do increase the total number of multisets, the specific combinatorial injection used to establish the lower bound remains valid. To eliminate any potential ambiguity, we will add an explicit paragraph and a small illustrative example in Section 4 of the revised manuscript. revision: partial

  2. Referee: [§5.2, Proposition 4] §5.2, Proposition 4 (non-existence): the proof invokes a finite-dimensional embedding and derives a contradiction via an infinite descending chain; the argument assumes the codomain is R^d with the standard product order, but does not explicitly rule out other finite-dimensional ordered vector spaces (e.g., with different cones), which is load-bearing for the claim that no MAS function exists at all.

    Authors: We thank the referee for highlighting this point. The definition of a MAS function in the paper requires the codomain to be R^d equipped with the standard product (componentwise) partial order. The non-existence argument in Proposition 4 constructs an infinite descending chain under this specific order to obtain a contradiction with finite dimensionality. Other partial orders on finite-dimensional vector spaces (induced by different cones) lie outside the MAS definition as introduced and are not needed for the neural models developed in the paper. We will revise the statement of Proposition 4 and the surrounding text to explicitly specify the product order and add a brief remark on why this is the relevant setting. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivations follow directly from the MAS definition

full rationale

The central results derive dimension bounds, non-existence for infinite ground sets, and the weakly-MAS neural construction strictly from the biconditional definition of MAS functions (S ⊆ T iff F(S) ≤ F(T)) and standard mathematical arguments on partial orders and embeddings. No steps reduce fitted parameters to predictions, smuggle ansatzes via self-citation, or import uniqueness theorems from the authors' prior work. The shift to a relaxed Holder-stable version for infinite sets is explicitly motivated as a workaround, keeping the derivation self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The work rests on standard definitions of partial orders on sets and on the existence of neural-network approximators; no free parameters or invented entities are introduced beyond the model architecture itself.

axioms (2)
  • standard math The natural partial order on sets is defined by subset inclusion.
    Invoked in the opening definition of MAS functions.
  • domain assumption Neural networks are universal approximators for continuous functions on compact domains.
    Used to claim that the MAS construction can approximate all monotone set functions.

pith-pipeline@v0.9.0 · 5748 in / 1547 out tokens · 45370 ms · 2026-05-21T20:01:47.670193+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

44 extracted references · 44 canonical work pages · 4 internal anchors

  1. [1]

    Hang Huang, J. M. Landsberg, and Jianfeng Lu. Geometry of backflow transformation ansatz for quantum many-body fermionic wavefunctions, 2021. URL https://arxiv.org/abs/ 2111.10314

    work page arXiv 2021

  2. [2]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 652–660, 2017

    work page 2017

  3. [3]

    How powerful are graph neural networks? InInternational Conference on Learning Representations, 2019

    Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? InInternational Conference on Learning Representations, 2019. URL https: //openreview.net/forum?id=ryGs6iA5Km

    work page 2019

  4. [4]

    Locality sensitive hashing in fourier frequency domain for soft set containment search

    Indradyumna Roy, Rishi Agarwal, Soumen Chakrabarti, Anirban Dasgupta, and Abir De. Locality sensitive hashing in fourier frequency domain for soft set containment search. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https: //openreview.net/forum?id=rUf0GV5CuU

    work page 2023

  5. [5]

    Explaining monotonic ranking functions

    Aman Singh, Naman Agarwal, Prateek Jain, and Manik Varma. Explaining monotonic ranking functions. InProceedings of the VLDB Endowment, volume 13, pages 2921–2934. VLDB Endowment, 2020

    work page 2020

  6. [6]

    Dessert: An efficient algorithm for vector set search with vector set queries.Advances in Neural Information Processing Systems, 36:67972–67992, 2023

    Joshua Engels, Benjamin Coleman, Vihan Lakshman, and Anshumali Shrivastava. Dessert: An efficient algorithm for vector set search with vector set queries.Advances in Neural Information Processing Systems, 36:67972–67992, 2023

    work page 2023

  7. [7]

    Theory of capacities.Annales de l’Institut Fourier, 5:131–295, 1953

    Gustave Choquet. Theory of capacities.Annales de l’Institut Fourier, 5:131–295, 1953. URL http://www.numdam.org/item/AIF_1953__5__131_0/. Foundational work introducing Choquet capacities and the Choquet integral

    work page 1953

  8. [8]

    Number 978-3-319-30690-2 in Theory and Decision Library C

    Michel Grabisch.Set Functions, Games and Capacities in Decision Making. Number 978-3-319-30690-2 in Theory and Decision Library C. Springer, March 2016. ISBN AR- RAY(0x694541d0). doi: 10.1007/978-3-319-30690-2. URL https://ideas.repec.org/ b/spr/thdlic/978-3-319-30690-2.html

    work page doi:10.1007/978-3-319-30690-2 2016

  9. [9]

    Monotone set functions and their generalizations.Fuzzy Sets and Systems, 393:1–19, 2020

    Hamzeh Agahi and Yong Ouyang. Monotone set functions and their generalizations.Fuzzy Sets and Systems, 393:1–19, 2020. doi: 10.1016/j.fss.2019.07.009. Generalizes monotonicity to fuzzy sets and hybrid integrals

    work page doi:10.1016/j.fss.2019.07.009 2020

  10. [10]

    Combinatorial auctions with decreas- ing marginal utilities

    Daniel Lehmann, Liadan Ita Lehmann, and Noam Nisan. Combinatorial auctions with decreas- ing marginal utilities. InProceedings of the 3rd ACM Conference on Electronic Commerce, pages 18–28. ACM, 2001

    work page 2001

  11. [11]

    Approximation algorithms for com- binatorial auctions with complement-free bidders

    Shahar Dobzinski, Noam Nisan, and Michael Schapira. Approximation algorithms for com- binatorial auctions with complement-free bidders. InProceedings of the 37th Annual ACM Symposium on Theory of Computing (STOC), pages 610–618. ACM, 2005

    work page 2005

  12. [12]

    Maximizing welfare when utility functions are subadditive.SIAM Journal on Computing, 39(1):122–142, 2009

    Uriel Feige. Maximizing welfare when utility functions are subadditive.SIAM Journal on Computing, 39(1):122–142, 2009

    work page 2009

  13. [13]

    Learning valuation functions

    Maria-Florina Balcan, Florin Constantin, Satoru Iwata, and Lei Wang. Learning valuation functions. InProceedings of the 25th Annual Conference on Learning Theory (COLT), pages 4.1–4.24, 2012

    work page 2012

  14. [14]

    Deep lattice networks and partial monotonic functions

    Haifeng You, Kevin Canini, Boyan Wang, Zhe Zhang, and Maya Gupta. Deep lattice networks and partial monotonic functions. InAdvances in Neural Information Processing Systems, pages 2981–2989, 2017

    work page 2017

  15. [15]

    Avrim Blum and Ronald L. Rivest. Training a 3-node neural network is np-complete. In Proceedings of the 1988 Workshop on Computational Learning Theory, pages 9–18, 1988

    work page 1988

  16. [16]

    Servedio

    Ryan O’Donnell and Rocco A. Servedio. Learning monotone functions from random exam- ples in polynomial time. InProceedings of the 35th Annual ACM Symposium on Theory of Computing, pages 448–456, 2003. 11

    work page 2003

  17. [17]

    Nader H. Bshouty. Exact learning boolean functions via the monotone theory.Information and Computation, 123(1):146–153, 1995

    work page 1995

  18. [18]

    Monotonic learning in the pac framework: A new perspective.arXiv preprint arXiv:2501.05493, 2025

    Ming Li, Chenyi Zhang, and Qin Li. Monotonic learning in the pac framework: A new perspective.arXiv preprint arXiv:2501.05493, 2025

    work page arXiv 2025

  19. [19]

    Partially ordered sets.American journal of mathematics, 63 (3):600–610, 1941

    Ben Dushnik and Edwin W Miller. Partially ordered sets.American journal of mathematics, 63 (3):600–610, 1941

    work page 1941

  20. [20]

    Deep Sets

    Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan Salakhutdinov, and Alexander Smola. Deep sets, 2018. URLhttps://arxiv.org/abs/1703.06114

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  21. [21]

    Neural injective func- tions for multisets, measures and graphs via a finite witness theorem

    Tal Amir, Steven Gortler, Ilai Avni, Ravina Ravina, and Nadav Dym. Neural injective func- tions for multisets, measures and graphs via a finite witness theorem. In A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors,Advances in Neu- ral Information Processing Systems, volume 36, pages 42516–42551. Curran Associates, Inc., 2023. ...

    work page 2023

  22. [22]

    On the limitations of representing functions on sets

    Edward Wagstaff, Fabian Fuchs, Martin Engelcke, Ingmar Posner, and Michael A Osborne. On the limitations of representing functions on sets. InInternational Conference on Machine Learning, pages 6487–6494. PMLR, 2019

    work page 2019

  23. [23]

    Polynomial width is suf- ficient for set representation with high-dimensional features.arXiv preprint arXiv:2307.04001, 2023

    Peihao Wang, Shenghao Yang, Shu Li, Zhangyang Wang, and Pan Li. Polynomial width is suf- ficient for set representation with high-dimensional features.arXiv preprint arXiv:2307.04001, 2023

    work page arXiv 2023

  24. [24]

    Fourier sliced-wasserstein embedding for multisets and measures,

    Tal Amir and Nadav Dym. Fourier sliced-wasserstein embedding for multisets and measures,

    work page

  25. [25]

    URLhttps://arxiv.org/abs/2405.16519

    work page arXiv

  26. [26]

    Permutation invariant representations with applications to graph deep learning.arXiv preprint arXiv:2203.07546, 2022

    Radu Balan, Naveed Haghani, and Maneesh Singh. Permutation invariant representations with applications to graph deep learning.arXiv preprint arXiv:2203.07546, 2022

    work page arXiv 2022

  27. [27]

    Fsw-gnn: A bi-lipschitz wl- equivalent graph neural network.arXiv preprint arXiv:2410.09118, 2024

    Yonatan Sverdlov, Yair Davidson, Nadav Dym, and Tal Amir. Fsw-gnn: A bi-lipschitz wl- equivalent graph neural network.arXiv preprint arXiv:2410.09118, 2024

    work page arXiv 2024

  28. [28]

    On the hölder stability of multiset and graph neural networks,

    Yair Davidson and Nadav Dym. On the hölder stability of multiset and graph neural networks,

    work page

  29. [29]

    URLhttps://arxiv.org/abs/2406.06984

    work page arXiv

  30. [30]

    A combinatorial problem in geometry.Compositio mathemat- ica, 2:463–470, 1935

    Paul Erdös and George Szekeres. A combinatorial problem in geometry.Compositio mathemat- ica, 2:463–470, 1935

    work page 1935

  31. [31]

    Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks

    Juho Lee, Yoonho Lee, Jungtaek Kim, Adam R. Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention-based permutation-invariant neural networks, 2019. URLhttps://arxiv.org/abs/1810.00825

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  32. [32]

    Monotonic networks.Advances in neural information processing systems, 10, 1997

    Joseph Sill. Monotonic networks.Advances in neural information processing systems, 10, 1997

    work page 1997

  33. [33]

    Neural estimation of submodular functions with applications to differentiable subset selection, 2022

    Abir De and Soumen Chakrabarti. Neural estimation of submodular functions with applications to differentiable subset selection, 2022. URLhttps://arxiv.org/abs/2210.11033

    work page arXiv 2022

  34. [34]

    Neural set function extensions: Learning with discrete functions in high dimensions

    Nikolaos Karalias, Joshua David Robinson, Andreas Loukas, and Stefanie Jegelka. Neural set function extensions: Learning with discrete functions in high dimensions. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors,Advances in Neural Information Processing Systems, 2022. URLhttps://openreview.net/forum?id=39XK7VJ0sKG

    work page 2022

  35. [35]

    3D ShapeNets: A Deep Representation for Volumetric Shapes

    Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes, 2015. URL https://arxiv.org/abs/1406.5670

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  36. [36]

    Munkres.Elementary differential topology, volume No

    James R. Munkres.Elementary differential topology, volume No. 54 ofAnnals of Mathematics Studies. Princeton University Press, Princeton, NJ, revised edition, 1966. Lectures given at Massachusetts Institute of Technology, Fall, 1961. 12

    work page 1966

  37. [37]

    Unconstrained monotonic neural networks, 2021

    Antoine Wehenkel and Gilles Louppe. Unconstrained monotonic neural networks, 2021. URL https://arxiv.org/abs/1908.05164

    work page arXiv 2021

  38. [38]

    The expressive power of neural networks: A view from the width, 2017

    Zhou Lu, Hongming Pu, Feicheng Wang, Zhiqiang Hu, and Liwei Wang. The expressive power of neural networks: A view from the width, 2017. URL https://arxiv.org/abs/1709. 02540

    work page 2017

  39. [39]

    Mini-batch consistent slot set encoder for scalable set encoding, 2021

    Bruno Andreis, Jeffrey Willette, Juho Lee, and Sung Ju Hwang. Mini-batch consistent slot set encoder for scalable set encoding, 2021. URLhttps://arxiv.org/abs/2103.01615

    work page arXiv 2021

  40. [40]

    Scalable set encoding with universal mini-batch consistency and unbiased full set gradient approximation, 2023

    Jeffrey Willette, Seanie Lee, Bruno Andreis, Kenji Kawaguchi, Juho Lee, and Sung Ju Hwang. Scalable set encoding with universal mini-batch consistency and unbiased full set gradient approximation, 2023. URLhttps://arxiv.org/abs/2208.12401

    work page arXiv 2023

  41. [41]

    Analyzing Inverse Problems with Invertible Neural Networks

    Lynton Ardizzone, Jakob Kruse, Sebastian Wirkert, Daniel Rahner, Eric W. Pellegrini, Ralf S. Klessen, Lena Maier-Hein, Carsten Rother, and Ullrich Köthe. Analyzing inverse problems with invertible neural networks, 2019. URLhttps://arxiv.org/abs/1808.04730. 13 APPENDIX 8 Proofs of the technical results 8.1 Proofs of existential results on MAS functions The...

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  42. [42]

    optimal coupling

    Now, Choosing δi := ti ∥ui∥ >0 gives us: P a⊤ui ≥δ i∥ui∥ ≥ 1 2 ,∀i∈[ℓ] . Since δi >0,∀i∈[ℓ] and ℓ is finite, we thus have: δ:= min i∈ℓ δi >0 . For this particular choice of δ, we have that P a⊤ui ≥δ∥u i∥ ≥ 1 2 ,∀i∈[ℓ] , and the observation is proved. . Now, consider all ℓ0 := N M M! vectors (xi −y τ(i))∈R d, where τ runs over all injective functions from ...

    work page

  43. [43]

    supp(σ ′)⊆[α, α+β] 2. R α+β α σ′(x) dx= 0 3.σ ′(x)≤C,∀x∈R 4.σ ′(x)≥c,∀x∈(α, α+γ·β) The above 4 conditions, along with σ(α) = 0 are the necessary-sufficienet conditions that character- izes the hat activation class. Proof. We first prove that, if σ belongs to the hat activation class as defined in 8, then conditions 1-4 are satisfied. Note that, if σ is a ...

    work page

  44. [44]

    support parameters

    Thus, the following hold for any ϵ >0 and any constant c∈R : Z α+γβ α σ′ 1(z)−h 1 θ(z) dz≤ϵand Z α+β α+γβ σ′ 2(t) +c·h 2 ϕ(t) dt≤ϵ(16) We now observe that, if x∈(−∞, α) , both the indicator functions: 1{α≤z≤α+γβ} and 1{α+γβ≤z≤α+β} as in the integrands of Equation (15) evaluate to 0, thus σΘ exactly coincides with σ. On the other hand, if x∈(α+β,∞) then we...

    work page