pith. sign in

arxiv: 2606.30822 · v1 · pith:CFWW5RFZnew · submitted 2026-06-29 · 📊 stat.ML · cs.IT· cs.LG· math.CV· math.IT

Separation Capacity of Scattering Networks

Konstantin H\"aberle , Helmut B\"olcskei This is my paper

Pith reviewed 2026-07-01 01:27 UTC · model grok-4.3

classification 📊 stat.ML cs.ITcs.LGmath.CVmath.IT
keywords separation capacityscattering networksCover's theorydichotomiesCNN feature extractorswavelet filterspooling operations
0
0 comments X

The pith

An extension of Cover's dichotomy counting shows that scattering network separation capacity is governed by its wavelet filters, layers, and pooling stages.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper extends Cover's function-counting theory to produce a formulation for separation capacity, defined as the number of binary label assignments a feature extractor can realize. It then applies the formulation to scattering networks to isolate the contributions of individual building blocks to that count. A sympathetic reader would care because the result supplies a combinatorial criterion for comparing and tuning these networks instead of relying only on end-to-end accuracy. The work closes with concrete design guidelines that follow from the identified governing factors.

Core claim

By first establishing a new, conceptually useful formulation for separation capacity within an extended Cover framework, the paper demonstrates that the separation capacity of scattering networks is controlled by the specific properties of their building blocks, including the choice of wavelet filters, the number and structure of layers, and the pooling operations.

What carries the argument

The extended formulation of separation capacity obtained by counting realizable dichotomies in the feature space produced by a scattering network.

If this is right

  • Scattering network designers can increase separation capacity by selecting wavelet filters and pooling stages that enlarge the set of realizable dichotomies.
  • The number of layers and paths in the network directly modulates the achievable separation capacity according to the derived expression.
  • Practical design rules for scattering networks follow immediately from the factors shown to govern the count of dichotomies.
  • Different scattering network architectures can be ranked by their separation capacity without training or testing on labeled data.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same extended counting approach could be applied to other fixed-feature CNN architectures that lack the scattering constraint.
  • If separation capacity correlates with downstream accuracy, it supplies an additional criterion for hyperparameter search in feature-extractor design.
  • The formulation might be used to compare scattering networks against learned convolutional layers on the basis of combinatorial capacity alone.

Load-bearing premise

The number of realizable dichotomies counted by the extended Cover framework serves as a useful proxy for how well a scattering network will perform as a feature extractor on actual classification tasks.

What would settle it

A scattering network configuration that the extended counting formula predicts will realize more dichotomies yet achieves lower accuracy on standard benchmark datasets than a configuration predicted to realize fewer dichotomies.

Figures

Figures reproduced from arXiv: 2606.30822 by Helmut B\"olcskei, Konstantin H\"aberle.

Figure 2.1
Figure 2.1. Figure 2.1: Dichotomy realized by a hyperplane through the origin, i.e., homogeneous linear separation. R 2 F+ F− Φ: R 2 → R 2 , f 7→  1 ∥f∥  R 2 [PITH_FULL_IMAGE:figures/full_fig_p003_2_1.png] view at source ↗
Figure 2.2
Figure 2.2. Figure 2.2: Spherical separating surface realized by application of a nonlinear transformation Φ followed by homogeneous linear separation in the feature space. linearly separable, as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p003_2_2.png] view at source ↗
Figure 2.8
Figure 2.8. Figure 2.8: Probability of separability. The gray dotted line corresponds to (2.1). dichotomies of F are Φ-separable. If there is no such N ∈ N, set SC (Φ) := 0. We call SC (Φ) the separation capacity of Φ. Thus, under the assumption that (L M) N F ∈ (R M) N : F is not in Φ-general position  = 0, for every N ∈ N, (2.2) we have, by the above discussion, that SC (Φ) is the largest N ∈ N such that C(N, M′ ) 2N = 2−N+… view at source ↗
Figure 3.1
Figure 3.1. Figure 3.1: Structure of a scattering network. The path (λ (j) 1 , λ(j) 2 , . . .) is indicated in blue. The outputs of each node are highlighted in red. Φ: CM → CM′ . The identification CM′ ≃ R 2M′ suggests that a dichotomy {F+, F−} of an N-point set F ⊂ CM is Φ-separable if there is a w ∈ CM′ such that3 ℜ(⟨Φ(f), w⟩) > 0, if f ∈ F+, ℜ(⟨Φ(f), w⟩) < 0, if f ∈ F−. Indeed, we have ℜ(⟨Φ(f), w⟩) = ⟨ℜ(Φ(f)), ℜ(w)⟩ + ⟨ℑ(Φ(… view at source ↗
Figure 4.1
Figure 4.1. Figure 4.1: Atoms of the Weyl–Heisenberg frame ΨWH, i.e., {χ} ∪ {gλ}λ∈Λ. of Section 3, we have the module sequence {(ΨWH, |·|2 ,Id)}n∈N. As noted in Remark 3.2, the modulus |·| is the traditional choice for the nonlinearity. Since |·| and |·|2 exhibit similar behavior, namely both show a demodulation and bandwidth doubling effect and result in a real-valued signal with conjugate symmetric spectrum (see, e.g., [29]),… view at source ↗
Figure 4.4
Figure 4.4. Figure 4.4: Computations in a node in the first layer. space. Now, by applying the modulus squared nonlinearity pointwise to the filtered signal f ∗ gλ, a bandwidth doubling effect occurs, as previously mentioned and illustrated in [PITH_FULL_IMAGE:figures/full_fig_p014_4_4.png] view at source ↗
Figure 4.9
Figure 4.9. Figure 4.9: Tree structure of every multi-layer network built from {(ΨWH, |·|2 ,Id)}n∈N, comprising only the nontrivial nodes. The gray part is superfluous as U[λ, 1] = U[λ, L], for every λ ∈ Λ. 1 k |(|f\∗ g1| 2)k|, (gb1)k 10−2 k |(U\[1, 1]f)k| [PITH_FULL_IMAGE:figures/full_fig_p016_4_9.png] view at source ↗
Figure 4.10
Figure 4.10. Figure 4.10: Computation of the feature maps associated with the path (1, 1). Proof. Thanks to Lemma 4.7 and using that the support sets of {χb} ∪ {gbλ}λ∈Λ are disjoint, (4.12) reads SC (Φ) = 4 · dimC [PITH_FULL_IMAGE:figures/full_fig_p016_4_10.png] view at source ↗
Figure 4.11
Figure 4.11. Figure 4.11: Computation of the feature maps associated with the path (1, L). 4.2 General case The example we just discussed naturally raises the questions of which module sequences yield a high separation capacity and what the driving and limiting factors are for achieving it. To ad￾dress these questions, we will now consider a general single-layer scattering network, constructed from the module sequence {(Ψ, ρ, P)… view at source ↗
Figure 5.1
Figure 5.1. Figure 5.1: Atoms of the wavelet frame Ψwvt satisfy (5.2) and hence the Littlewood–Paley condition (5.1). {(2j mod M): j ≥ 0} if M is odd. The Littlewood–Paley condition A∥f∥ 2 ≤ ∥f ∗ χ∥ 2 + X λ∈Λ ∥f ∗ gλ∥ 2 ≤ B∥f∥ 2 , for all f ∈ C M, (5.1) with 0 < A ≤ B < ∞, holds if and only if ϕ, ψ are such that [PITH_FULL_IMAGE:figures/full_fig_p025_5_1.png] view at source ↗
read the original abstract

In this paper, we attempt to enhance the theoretical understanding of convolutional neural networks (CNNs) as feature extractors in classification tasks by analyzing them through the lens of Cover's function-counting theory. Specifically, our focus lies on the notion of separation capacity, a combinatorial quantity derived from counting the number of realizable dichotomies (i.e., binary label assignments). Our contributions are threefold. First, we extend Cover's framework by establishing a conceptually insightful and practically useful formulation for the separation capacity. Second, leveraging this formulation, we identify the factors governing the separation capacity of feature extractors that employ a specific CNN architecture, so-called scattering networks, in terms of their network building blocks. Third, we provide practical insights for scattering network design.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The paper extends Cover's function-counting theory to define a formulation for the separation capacity of scattering networks used as CNN feature extractors. It claims this formulation identifies the network building blocks (layers, scales, wavelets) that govern the number of realizable dichotomies and yields practical design insights for scattering networks.

Significance. If the extended formulation is correct and the dichotomy count proves a reliable proxy, the work could supply a combinatorial tool for analyzing and designing scattering networks. The paper's strength would lie in making the Cover-style count actionable for this architecture, but the significance is reduced by the lack of demonstrated connection between the combinatorial quantity and actual classification performance.

major comments (1)
  1. [Abstract and §3] The central claim (abstract and §3) that the formulation identifies governing factors for separation capacity is load-bearing only if the number of realizable dichotomies serves as a faithful proxy for feature quality in classification. The manuscript does not address how correlations among scattering paths sharing the same mother wavelet, the deliberate invariance properties, or dependence on the data measure affect this proxy; without such justification or empirical checks, the identified factors may not govern practical separation performance.
minor comments (1)
  1. [§2] Notation for the extended Cover formulation could be introduced more explicitly with a dedicated equation block early in the text.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We appreciate the referee's detailed feedback on our manuscript. We respond to the major comment below, clarifying the scope of our theoretical contribution.

read point-by-point responses

  1. Referee: [Abstract and §3] The central claim (abstract and §3) that the formulation identifies governing factors for separation capacity is load-bearing only if the number of realizable dichotomies serves as a faithful proxy for feature quality in classification. The manuscript does not address how correlations among scattering paths sharing the same mother wavelet, the deliberate invariance properties, or dependence on the data measure affect this proxy; without such justification or empirical checks, the identified factors may not govern practical separation performance.

    Authors: The separation capacity in our paper is defined as the number of realizable dichotomies, extending Cover's combinatorial framework. Our formulation provides a closed-form expression that explicitly depends on the network's layers, scales, and wavelets, thereby identifying these as the governing factors for this capacity. Cover's theory counts dichotomies under the assumption of general position in the feature space, without explicitly modeling path correlations or data measure dependence. The deliberate invariance in scattering networks is accounted for in the feature map definition, but the count remains combinatorial. While we agree that connecting this to empirical classification performance would strengthen the practical insights, the current work focuses on the theoretical count. We can revise the manuscript to include a discussion of these assumptions and their implications for the proxy in a dedicated limitations subsection. revision: partial

Circularity Check

0 steps flagged

No significant circularity; derivation extends external Cover framework to scattering networks

full rationale

The paper extends Cover's existing function-counting theory with a new formulation for separation capacity, then applies the result to identify governing factors in scattering networks' building blocks (layers, scales, wavelets). No equations or steps reduce by construction to self-defined quantities, fitted parameters renamed as predictions, or load-bearing self-citations whose content is unverified. The derivation remains self-contained against the external Cover benchmark and the independent definition of scattering networks; the proxy link to real classification performance is a modeling assumption outside the circularity criteria.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review yields no identifiable free parameters, axioms, or invented entities; the approach rests on the unexamined premise that Cover's dichotomy counting applies directly to the scattering architecture.

pith-pipeline@v0.9.1-grok · 5656 in / 1017 out tokens · 37838 ms · 2026-07-01T01:27:59.725332+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Function-Counting Theory for Low-Dimensional Data Structures

    stat.ML 2026-07 unverdicted novelty 6.0

    Refines Cover's dichotomy counts and separation capacity for low-dimensional data by adjusting the general position assumption.

Reference graph

Works this paper leans on

36 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [1]

    C. M. Bishop and N. M. Nasrabadi,Pattern recognition and machine learning. Springer, 2006, vol. 4, no. 4

    2006

  2. [2]

    Support-vector networks,

    C. Cortes and V. Vapnik, “Support-vector networks,”Machine learning, vol. 20, pp. 273– 297, 1995

    1995

  3. [3]

    Large-scale learning with SVM and convolutional for generic object categorization,

    F. J. Huang and Y. LeCun, “Large-scale learning with SVM and convolutional for generic object categorization,” in2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 1. IEEE, 2006, pp. 284–291

    2006

  4. [4]

    Group invariant scattering,

    S. Mallat, “Group invariant scattering,”Communications on Pure and Applied Mathemat- ics, vol. 65, no. 10, pp. 1331–1398, 2012

    2012

  5. [5]

    Invariant scattering convolution networks,

    J. Bruna and S. Mallat, “Invariant scattering convolution networks,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1872–1886, 2013

    2013

  6. [6]

    Deep scattering spectrum,

    J. And´ en and S. Mallat, “Deep scattering spectrum,”IEEE Transactions on Signal Pro- cessing, vol. 62, no. 16, pp. 4114–4128, 2014

    2014

  7. [7]

    A mathematical theory of deep convolutional neural net- works for feature extraction,

    T. Wiatowski and H. B¨ olcskei, “A mathematical theory of deep convolutional neural net- works for feature extraction,”IEEE Transactions on Information Theory, vol. 64, no. 3, pp. 1845–1866, 2018

    2018

  8. [8]

    Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition,

    T. M. Cover, “Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition,”IEEE Transactions on Electronic Computers, no. 3, pp. 326–334, 1965

    1965

  9. [9]

    A practical guide to support vector classification,

    C.-W. Hsu, C.-C. Chang, and C.-J. Lin, “A practical guide to support vector classification,” Taipei, Taiwan, 2003

    2003

  10. [10]

    Bounds on the learning capacity of some multi-layer net- works,

    G. Mitchison and R. Durbin, “Bounds on the learning capacity of some multi-layer net- works,”Biological Cybernetics, vol. 60, no. 5, pp. 345–365, 1989

    1989

  11. [11]

    Separating capacity of analytic neurons,

    A. Kowalczyk, “Separating capacity of analytic neurons,” inProceedings of 1994 IEEE International Conference on Neural Networks (ICNN’94), vol. 5. IEEE, 1994, pp. 3038– 3043. Separation Capacity of Scattering Networks 35

    1994

  12. [12]

    On the uniform convergence of relative frequencies of events to their probabilities,

    V. N. Vapnik and A. Y. Chervonenkis, “On the uniform convergence of relative frequencies of events to their probabilities,”Theory of Probability & Its Applications, vol. 16, no. 2, pp. 264–280, 1971

    1971

  13. [13]

    Shalev-Shwartz and S

    S. Shalev-Shwartz and S. Ben-David,Understanding machine learning: From theory to algorithms. Cambridge university press, 2014

    2014

  14. [14]

    Function-counting theory for low-dimensional data struc- tures,

    K. H¨ aberle and H. B¨ olcskei, “Function-counting theory for low-dimensional data struc- tures,”in preparation, 2026

    2026

  15. [15]

    S. G. Krantz and H. R. Parks,A primer of real analytic functions. Springer Science & Business Media, 2002

    2002

  16. [16]

    The zero set of a real analytic function,

    B. Mityagin, “The zero set of a real analytic function,”Mathematical Notes, vol. 107, no. 3, pp. 529–530, 2020

    2020

  17. [17]

    Arrhythmia classification of 12-lead electrocardiograms by hybrid scattering-lstm networks,

    P. A. Warrick, V. Lostanlen, M. Eickenberg, J. And´ en, and M. N. Homsi, “Arrhythmia classification of 12-lead electrocardiograms by hybrid scattering-lstm networks,” in2020 Computing in Cardiology. IEEE, 2020, pp. 1–4

    2020

  18. [18]

    Maximum-entropy scattering models for financial time series,

    R. Leonarduzzi, G. Rochette, J.-P. Bouchaud, and S. Mallat, “Maximum-entropy scattering models for financial time series,” inICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 5496–5500

    2019

  19. [19]

    Scale dependencies and self-similar models with wavelet scattering spectra,

    R. Morel, G. Rochette, R. Leonarduzzi, J.-P. Bouchaud, and S. Mallat, “Scale dependencies and self-similar models with wavelet scattering spectra,”arXiv preprint arXiv:2204.10177, 2022

    work page arXiv 2022

  20. [20]

    Solid harmonic wavelet scattering: Predicting quantum molecular energy from invariant descriptors of 3d electronic densities,

    M. Eickenberg, G. Exarchakis, M. Hirn, and S. Mallat, “Solid harmonic wavelet scattering: Predicting quantum molecular energy from invariant descriptors of 3d electronic densities,” Advances in Neural Information Processing Systems, vol. 30, 2017

    2017

  21. [21]

    A new approach to observational cosmol- ogy using the scattering transform,

    S. Cheng, Y.-S. Ting, B. M´ enard, and J. Bruna, “A new approach to observational cosmol- ogy using the scattering transform,”Monthly Notices of the Royal Astronomical Society, vol. 499, no. 4, pp. 5902–5914, 2020

    2020

  22. [22]

    Towards an optimal estimation of cosmological parameters with the wavelet scattering transform,

    G. Valogiannis and C. Dvorkin, “Towards an optimal estimation of cosmological parameters with the wavelet scattering transform,”Physical Review D, vol. 105, no. 10, p. 103534, 2022

    2022

  23. [23]

    Scattering spectra models for physics,

    S. Cheng, R. Morel, E. Allys, B. M´ enard, and S. Mallat, “Scattering spectra models for physics,”PNAS nexus, vol. 3, no. 4, pp. 1–13, 2024

    2024

  24. [24]

    Christensen,An introduction to frames and Riesz bases

    O. Christensen,An introduction to frames and Riesz bases. Springer, 2003, vol. 7

    2003

  25. [25]

    Kaiser and L

    G. Kaiser and L. H. Hudgins,A friendly guide to wavelets. Springer, 1994, vol. 300

    1994

  26. [26]

    Continuous frames in Hilbert space,

    S. T. Ali, J.-P. Antoine, and J.-P. Gazeau, “Continuous frames in Hilbert space,”Annals of physics, vol. 222, no. 1, pp. 1–37, 1993

    1993

  27. [27]

    Daubechies,Ten lectures on wavelets

    I. Daubechies,Ten lectures on wavelets. SIAM, 1992

    1992

  28. [28]

    Necessary and sufficient conditions for discrete wavelet frames inC N,

    Deepshikha and L. K. Vashisht, “Necessary and sufficient conditions for discrete wavelet frames inC N,”Journal of Geometry and Physics, vol. 117, pp. 134–143, 2017

    2017

  29. [29]

    Topology reduction in deep convolutional feature extraction networks,

    T. Wiatowski, P. Grohs, and H. B¨ olcskei, “Topology reduction in deep convolutional feature extraction networks,” inWavelets and Sparsity XVII, vol. 10394. SPIE, 2017, pp. 269–280

    2017

  30. [30]

    Gr¨ ochenig,Foundations of time-frequency analysis

    K. Gr¨ ochenig,Foundations of time-frequency analysis. Birkh¨ auser, 2001. 36 K. H¨ aberle and H. B¨ olcskei

    2001

  31. [31]

    Hall,The theory of groups

    M. Hall,The theory of groups. Courier Dover Publications, 2018

    2018

  32. [32]

    Submersions and preimages of sets of measure zero,

    S. P. Ponomarev, “Submersions and preimages of sets of measure zero,”Siberian Mathe- matical Journal, vol. 28, no. 1, pp. 153–163, 1987

    1987

  33. [33]

    Rudin,Functional analysis

    W. Rudin,Functional analysis. McGraw-Hill, 1991

    1991

  34. [34]

    Narasimhan,Several complex variables

    R. Narasimhan,Several complex variables. University of Chicago Press, 1971

    1971

  35. [35]

    Energy propagation in deep convolutional neural networks,

    T. Wiatowski, P. Grohs, and H. B¨ olcskei, “Energy propagation in deep convolutional neural networks,”IEEE Transactions on Information Theory, vol. 64, no. 7, pp. 4819–4842, 2018

    2018

  36. [36]

    Zhang,Matrix theory: Basic results and techniques

    F. Zhang,Matrix theory: Basic results and techniques. Springer, 2011

    2011