{"id":"827a83f3-ce6e-45f7-a027-7ad5dc17d589","arxiv_id":"2505.09710","paper_version":4,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"low","formal_verification":"none","parameter_count":4,"one_line_summary":"Pure max-plus/min-plus deep networks are shown to be the opposite of universal approximators, while inserting O(N)-parameter linear scalings per layer yields provably universal morphological networks.","lead":"Pure deep morphological networks that only use max-plus and min-plus operations are shown to be unable to approximate arbitrary continuous functions, because their gradients are almost always single coordinate vectors or zero. The authors fix this by inserting constrained linear scalings between morphological layers, and prove that the resulting Max-Plus-Min and Hybrid-MLP networks are universal approximators on compact domains.","discovery_kind":"first_principles","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 4's proof invokes, without stating or proving, the density of max-plus-min affine combinations; the fact is true and recoverable, but as written the MPM universality result rests on an unstated lemma.","rationale":"The reader's weakest assumption identifies exactly the point where Theorem 4 is least secure. The proof in Appendix B shows constructively that MPM can represent every function of the form max_k(a_k^T x + b_k) + min_m(c_m^T x + d_m), but the final universality step depends on the density of that class in C(K), which is asserted only as a 'byproduct' of Maxout universality. This is a genuine gap in presentation: it should be a stated lemma. However, the gap is easily closed, because Maxout universal approximation implies max-affine functions are dense, and the max-plus-min class contains max-affine functions by taking the min term to be identically zero. The MPM construction can realize that zero term through an all-zero affine function. Thus the central mathematical claim is correct, and the concern is about self-containedness rather than soundness. I also checked Theorem 5: its construction replaces ReLU and maxout layers by morphological layers with explicit constants bounded via Lemmas B.1 and B.2, and the argument is sound. The negative results in Theorems 1-3 are supported by the gradient lemmas and Lemma A.2. The empirical issues flagged by the reader (the Fashion-MNIST 'zero drop' phrasing and the missing notebooks) are secondary and do not affect the theoretical core. Therefore the reader's CONDITIONAL verdict is appropriate, with the condition being to add and prove the missing density lemma.","tokens_in":1983,"tokens_out":2708,"duration_ms":211805,"concrete_test":"Independently state and prove the missing density lemma in Appendix B: for compact K subset of R^d, the class {max_{k in [K]}(a_k^T x + b_k) + min_{m in [M]}(c_m^T x + d_m)} is dense in C(K). Verify the proof uses only the Maxout universal approximation theorem plus the zero min-affine term, and confirm the MPM construction in Appendix B can realize that zero term (set all parallel affine coefficients to 0). If the lemma cannot be proved from these ingredients, or if the zero min term is not realizable by the MPM recursion, Theorem 4's conclusion is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Appendix B, proof of Theorem 4, asserts that 'a byproduct of the proof of universality of Maxout networks' is the density of the class F = {max_{k in [K]}(a_k^T x + b_k) + min_{m in [M]}(c_m^T x + d_m)} in C(K) for compact K. This is the step that converts the paper's constructive result—every MPM can compute a function in F—into the universal approximation claim. The assertion is not stated as a lemma and no proof or precise citation is given. The concern is load-bearing because if F were not dense, the construction would only show MPM can represent a restricted subclass of continuous functions. In this case the assertion is true: by the Maxout universal approximation theorem, max-affine functions are dense in C(K), and the class F contains max-affine functions by taking M = 1, c_1 = 0, d_1 = 0 (the zero min-affine term, which the MPM construction can produce). So the gap is a completeness/reproducibility gap in the proof as written, not a mathematical falsehood. It should be fixed by making the lemma explicit and proving it; until then Theorem 4 is not fully self-contained.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies deep morphological neural networks (DMNNs) built from max-plus and min-plus morphological perceptrons. It proves that existing 'pure' DMNNs that use only max-plus/min-plus MPs are not universal approximators (Theorem 1) and that DEP-based networks are not universal approximators (Theorem 3), arguing that their output gradients are almost everywhere restricted to zero or a single coordinate unit vector (respectively, nonnegative with L1 norm at most one). It then proposes new architectures—MPM, RMPM, MPM-SVD, and Hybrid-MLP—that insert linear activations between morphological layers, and proves universal approximation theorems for MPM (Theorem 4) and Hybrid-MLP (Theorem 5). The experiments demonstrate trainability, pruning advantages, and convergence behavior with large batches.","tokens_in":40418,"tokens_out":14342,"duration_ms":132894,"significance":"The paper makes a clear theoretical contribution by identifying algebraic and analytic obstructions to universality in morphological networks, and by providing constructive architectures that overcome these obstructions while keeping the number of learnable activation parameters modest. The non-universality proofs are rigorous and the universality proofs are constructive, which is valuable for the morphological deep learning community. The empirical results support the claims of trainability and strong pruning properties, which are relevant for edge computing; if the theoretical results hold, they settle a question in the literature about the expressivity of 'pure' morphological networks.","major_comments":[{"comment":"The proof asserts, without stating a lemma or giving a proof or precise citation, that the class of functions of the form max_{k in [K]}(a_k^T x + b_k) + min_{m in [M]}(c_m^T x + d_m) is dense in C(K) for compact K, calling it 'a byproduct of the proof of universality of Maxout networks.' This density claim is load-bearing: it is exactly what converts the constructive result that every MPM can compute such a function into the universal approximation statement. The claim is true (for example, the class contains max-affine functions by taking M=1, c=0, d=0, and max-affine functions are dense by the Maxout universality theorem), but as written Theorem 4 is not fully self-contained. Please state and prove this density fact explicitly as a lemma, or cite a specific theorem from which it follows directly.","section":"Appendix B, proof of Theorem 4"}],"minor_comments":[{"comment":"The text claims 'zero accuracy drop' for the 1173-parameter pruned MPM, but on Fashion-MNIST the unpruned MPM (Table 2) reports 82.86±0.17 while the SNIP-pruned network (Table 4) reports 82.73±0.33, a drop of 0.13 percentage points. Even if this is within one standard deviation, the phrase 'zero accuracy drop' is inaccurate; please rephrase to 'negligible accuracy drop' or specify the exact comparison being made.","section":"Section 4.2, Tables 2 and 4"},{"comment":"The appendix explicitly states that 'Most of the claims in this appendix are qualitative and lack formal proofs.' Since the main text (Section 3) refers to this appendix to justify design choices such as setting lambda = 1/2, please add a sentence in the main text indicating that those arguments are heuristic, so that readers do not mistake them for proved results.","section":"Appendix C"},{"comment":"In the construction of the second layer, the bias w^(2)_{(d-1)0} for the output that sums two terms is not specified; the reader must infer that it is set to 0 or another value that makes the displayed equalities hold. Please specify all weights explicitly for that unit.","section":"Appendix B, proof of Theorem 4"},{"comment":"The abstract says 'Only O(N) parameters (or learnable parameters) per layer of size N belong to the activations,' but in Setting 2 the activation includes fixed O(N^2) matrices U and V; only the learnable parameters are O(N). Please clarify this distinction in the abstract and in the table of parameter counts.","section":"Throughout"},{"comment":"The reference for the bound on the expectation of the maximum of Gaussian random variables is a Mathematics Stack Exchange answer (Sivaraman, 2011), which is not a standard archival source; consider replacing it with a textbook reference or a brief self-contained derivation.","section":"References"},{"comment":"The universal approximation theorems are stated for the class of all MPM (resp. Hybrid-MLP) networks, but the proofs construct networks whose depth grows with the input dimension and the number of affine pieces. Please add a sentence clarifying that the density claim is over architectures of arbitrary depth and width, not over a fixed architecture, to avoid a potential misinterpretation.","section":"Section 3, Theorems 4 and 5"}],"recommendation":"major_revision","confidential_remarks":"The main theorems appear to be correct, and the only substantial gap in the theory is the unstated density lemma in the proof of Theorem 4, which is true and easily fixed. The experimental work is extensive, though the 'zero accuracy drop' phrasing should be corrected. The novelty claims about being the first to train fully morphological networks on CIFAR-10 are plausible but difficult to verify; they are not a barrier to publication. Overall the paper is a solid contribution that needs a focused revision rather than a reworking of the central ideas."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The negative theorems are the real contribution here, and they are solid. Pure max-plus/min-plus networks have gradients that are a.e. either 0 or a coordinate unit vector (Theorem 1), DEP networks have gradients that are a.e. nonnegative with L1 norm at most 1 (Theorem 3), and both consequences — non-universality on compact domains and the sparse-gradient training problem (Theorem 2) — are proved carefully and self-containedly in Appendix A. The classical collapse of stacked max-plus layers under tropical associativity is explained well and ties the narrative together. The positive half is mostly right too: the MPM construction in Appendix B that builds affine terms, sums them, and then combines them via max-plus/min-plus is genuinely constructive, and the simulation of ReLU and maxout networks in Theorem 5 with the norm bounds in Lemmas B.1/B.2 is rigorous.\n\nThe one real soft spot is Theorem 4's proof. It invokes the density in C(K) of functions max_k(a_k^T x + b_k) + min_m(c_m^T x + d_m) as \"a byproduct of the proof of universality of Maxout networks,\" without stating it as a lemma or proving it. That step is load-bearing: without it, the construction only shows MPM can represent that specific class, not that it approximates everything continuous. The claim is true and easily fixable — the class contains max-affine functions by taking M = 1 with a zero min-term, and maxout universality supplies the rest — so this is a completeness gap in the written proof, not a mathematical error. It should be patched before publication.\n\nMinor issues in proportion: the \"zero accuracy drop\" pruning claim is slightly overconfident for Fashion-MNIST (82.73% pruned versus 82.86% unpruned is a small but nonzero drop); the appendix promises python notebooks that are not actually linked anywhere; and the initialization needed some trial and error, which is honestly disclosed but leaves a few free parameters in the experiments. The citation practice looks fair to me — the relevant morphological literature is covered, and the authors' own past work is cited where it is actually the relevant source.\n\nWorth a serious referee. I would send it out, and ask for the density lemma to be stated and proved, the pruning sentence reworded, and the code link added. Otherwise the theory holds up.","headline":"Solid negative theorems on pure morphological networks; the positive universality results are right but Theorem 4 needs an explicit density lemma stated and proved.","tokens_in":40954,"tokens_out":4090,"would_cite":true,"duration_ms":39487,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Deep networks built purely from max-plus and min-plus perceptrons cannot approximate all continuous functions, but adding learnable linear scalings between morphological layers restores universal approximation with only $O(N)$ extra…","keywords":["morphological neural networks","max-plus algebra","min-plus algebra","universal approximation","Dilation-Erosion Perceptron","Maxout networks","gradient sparsity","network pruning"],"falsifier":"Compute the input gradient of any pure max-plus/min-plus network at a point where every maximizer and minimizer is unique: any gradient other than $0$ or a standard basis vector refutes Theorem 1. For the MPM claim, try to approximate $2x_1$ uniformly on $[-1,1]^d$ with finite-width MPM networks; if no width reaches error below $1/2$, Theorem 4 is false, whereas success is expected because the max-plus-min affine class is a standard difference-of-convex dense family.","tokens_in":39953,"feed_emoji":"🧮","tokens_out":15742,"duration_ms":134431,"temperature":0.7,"pith_summary":"The paper argues that depth alone does not make a morphological network powerful. It proves that a network built only from max-plus and min-plus morphological perceptrons has gradients that are almost everywhere either zero or a single coordinate unit vector, so it cannot even represent a sloping linear function, let alone approximate every continuous function. The paper then proposes two architectures---the Max-Plus-Min network and the Hybrid-MLP---that place learnable linear operations between morphological layers, and proves both are universal approximators on compact domains with only $O(N)$ learnable activation parameters per layer. Experiments on MNIST, Fashion-MNIST, and CIFAR-10 show the resulting networks train, generalize, and, under pruning, degrade less than linear networks. The upshot is a design rule with practical weight: morphological networks need a small amount of linearity, not more morphology, to become expressive.","feed_headline":"Pure morphological networks need linear activations to be universal","feed_subtitle":"Adding one learnable scale after a summed dilation and erosion restores universal approximation with only O(N) extra parameters.","key_machinery":"The load-bearing object is the Max-Plus-Min (MPM) morphological layer. For an input $x$ it computes $(w_{i0}\\vee \\max_j (x_j+w_{ij}))+(m_{i0}\\wedge \\min_j (x_j+w_{ij}))$, a dilation and an erosion that share weights but have different biases, and then multiplies each output by a learnable scalar $\\alpha_i$; the MPM-SVD variant instead applies a learnable diagonal matrix between two fixed random orthogonal matrices. The universality proof works by choosing a large constant $C$: weights near $+C$, $-C$, or $0$ make each input feed the maximum branch, the minimum branch, or neither, so the layer can first build affine functions $a^\\top x+b$ and then combine them into $\\max_k(a_k^\\top x+b_k)+\\min_m(c_m^\\top x+d_m)$. Because this last class is dense in the continuous functions---the paper takes this from the Maxout universality argument---the architecture inherits universal approximation.","core_discovery":"The central claim is a pair of obstruction and recovery theorems. Any single-output network that only composes max-plus and min-plus morphological perceptrons is Lipschitz and, almost everywhere, its input gradient is either $0$ or a standard basis vector $e_i$; hence its functions lie in a class that cannot be dense in the continuous functions, and the same sparsity limits training through the parameters (Theorems 1 and 2). DEP-style blends of dilation and erosion are also non-universal: their gradients are componentwise nonnegative with $\\ell^1$ norm at most $1$, which again excludes simple linear maps such as $2x_1$ (Theorem 3). The proposed Max-Plus-Min layer---a max-plus dilation plus a min-plus erosion sharing weights, with separate biases, followed by a learnable scaling---overcomes the obstruction, and the paper proves the resulting MPM network and the Hybrid-MLP are universal approximators on compact domains (Theorems 4 and 5). In the Hybrid-MLP case the proof is explicit: any fully connected ReLU or Maxout network is a special case of the Hybrid-MLP.","pith_inferences":["The unstated density fact behind Theorem 4 is the statement that functions $\\max_k(a_k^\\top x+b_k)+\\min_m(c_m^\\top x+d_m)$ are dense, which is the difference-of-convex (DC) property; a direct DC-density proof would make the MPM argument self-contained.","The paper's initialization analysis predicts a testable pattern: morphological weights should be initialized near a common mean while learnable scalings absorb variance, and ablating initialization over mean and standard deviation should show this explicitly.","The large-batch requirement of the Hybrid-MLP suggests morphological layers inject high-variance gradient noise; variance-reduced optimizers or a smaller effective mini-batch may restore trainability at batch 64.","If diagonal scaling suffices for universality in MPM, similar summed-dilation-erosion layers with cheap linear couplings may make convolutional morphological networks universal approximators while keeping the pruning advantage."],"forward_implications":["Pure max-plus networks of depth $L\\ge 2$ collapse, in the semiring, to a single morphological layer, so any expressive fully morphological network needs some linear operation inserted between morphological layers.","The MPM and MPM-SVD networks are universal approximators on compact domains while keeping activation parameters at $O(N)$ per layer, so the sparsity and prunability of morphological networks is not lost.","The Hybrid-MLP is a strict generalization of ReLU and Maxout MLPs, and with large batches it converges faster than a standard MLP or Maxout network on MNIST and Fashion-MNIST.","Morphological networks are substantially more prunable than linear networks: at 90% unstructured $\\ell^1$ pruning, MPM keeps about 92% test accuracy on MNIST while the MLP collapses to about 38%, and SNIP can prune MPM to 1173 parameters on MNIST with no accuracy drop.","Residual connections and weight dropout improve generalization, with RMPM-Drop reaching 97.49% test accuracy on MNIST, within 0.52 points of a standard MLP."],"supporting_citations":[{"why":"Supplies the Maxout universality result whose byproduct the MPM proof uses to assert that max-of-affine plus min-of-affine functions are dense.","marker":"Goodfellow et al., 2013"},{"why":"Defines deep morphological hybrid networks and the claim that morphological layers are nonlinear enough to need no activations, which Theorems 1-3 target.","marker":"Franchi et al. (2020)"},{"why":"Introduces the DEP update with lambda, weights, and sigmoid activation that Theorem 3 and its activated variant analyze.","marker":"de A. Araújo et al., 2017"},{"why":"Shows two max-plus MP layers reduce to one layer and reports prunability of morphological networks, used as background and as a pruning-comparison baseline.","marker":"Dimitriadis and Maragos (2021)"},{"why":"Representation Theorem for increasing translation-invariant filters as suprema of erosions, which Theorem 1 complements by delimiting what finite max-plus/min-plus networks can express.","marker":"Maragos and Schafer (1987)"}],"fun_headline_variants":["Morphological nets fail universality without linear activations","Linear activations unlock universal approximation in morphological nets","Pure morphological nets can't approximate all functions—add scaling","Learnable scaling makes morphological networks universal approximators","Gradient sparsity blocks pure morphological networks—fix with linear maps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The MPM universality proof rests on the unstated assumption that functions of the form $\\max_k(a_k^\\top x+b_k)+\\min_m(c_m^\\top x+d_m)$ can approximate any continuous function on the compact domain; the paper cites this as a byproduct of Maxout universality but never states or proves it as a lemma.","fun_headline_variants_meta":{"raw":{"variants":["Morphological nets fail universality without linear activations","Linear activations unlock universal approximation in morphological nets","Pure morphological nets can't approximate all functions—add scaling","Learnable scaling makes morphological networks universal approximators","Gradient sparsity blocks pure morphological networks—fix with linear maps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000207,"raw_usage":{"total_tokens":1388,"prompt_tokens":918,"completion_tokens":470,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":534,"completion_tokens_details":{"reasoning_tokens":391}},"tokens_in":534,"tokens_out":470,"duration_ms":4453,"temperature":1.0,"reasoning_tokens":391,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:28:00.632015+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the input gradient of any pure max-plus/min-plus network at a point where every maximizer and minimizer is unique: any gradient other than $0$ or a standard basis vector refutes Theorem 1. For the MPM claim, try to approximate $2x_1$ uniformly on $[-1,1]^d$ with finite-width MPM networks; if no width reaches error below $1/2$, Theorem 4 is false, whereas success is expected because the max-plus-min affine class is a standard difference-of-convex dense family.","supporting_citations":[],"review_version":1}