On high probability of universal approximation in random basis expansions with non-continuous weight sampling

John E. Darges

REVIEW 4 major objections 4 minor 16 references

On high probability of universal approximation in random basis expansions with non-continuous weight sampling

T0 review · 4 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Random basis expansions with non-continuous dense-support weight sampling and complex-valued activations approximate any continuous function on compact sets with arbitrarily high probability, governed by a logarithmic sample-count bound.

desk verdict The advertised complex case is not defined, but the real-input transfer idea is worth a repair. read the letter →

arxiv 2607.16551 v1 pith:BEQCTYUU submitted 2026-07-17 math.FA math.PR

John E. Darges This is my paper · ORCID

classification math.FAmath.PR MSC 41A3068T07

keywords randombasisexpansionuniversalapproximationdensesupportnon-continuousweightsamplingcomplex-valuedactivationhighprobabilityspike-and-slabfeatures

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to extend universal approximation results for random basis expansions to two untested settings: weight distributions that are not continuous but have dense support, and activation functions that are complex-valued. Its main theorem claims that if the weights and biases are sampled iid from such a distribution, then for any continuous target on a compact set and any accuracy, a random-basis expansion with a sufficiently large—and logarithmically bounded—number of samples is within that accuracy with probability at least 1−η. The practical payoff is that sparse and spike-and-slab sampling schemes, which current theory does not cover, would inherit the same density guarantee as continuous sampling, and complex-valued feature maps used in random Fourier methods would also qualify. The proof works by taking a known approximating expansion, sampling weights into small neighborhoods of its component weights, and union-bounding the chance that any neighborhood is missed.

What carries the argument

The central object is the random basis expansion (1), a single-layer network with random hidden weights and tunable output coefficients. The argument's engine is Lemma 2.4, a continuity result: as weights (ω_j,b_j) converge in R^{n+1}, the induced feature functions converge in L∞ on compact sets, provided φ is continuous. This lets the proof transplant universal approximation from the fully-trained setting: for each term of a reference expansion, the probability that a sampled weight lands in a δ-ball around the reference weight is positive because the sampling distribution has dense support; a union bound over the q reference terms yields the sample-count formula m ≥ log(...). The complex c

What would settle it

Take any continuous non-polyharmonic φ:R→C and any point x∈C^n with ωᵀx+b∉R; since φ is only defined on R, the basis function is undefined at that point. Placing such a point in the compact set K shows the theorem's statement cannot even be evaluated as written, so the claim fails for the stated domain.

Watch

Extended reading notes

Core claim

In the paper's own terms, the central discovery is Theorem 3.1: for any continuous f:C^n→C, any continuous non-polyharmonic φ:R→C, and any iid sample distribution D with dense support in R^{n+1}, for every compact K⊂C^n and ε>0 there is an m (logarithmic in 1/η) such that with probability at least 1−η the expansion Σⱼ αⱼ φ(ωⱼᵀx+bⱼ) lies within ε of f in L∞(K). The coefficients are chosen by matching sampled weights to the weights of a q-term expansion guaranteed by classical universal approximation, zeroing the rest. The theorem is an existence result—it does not construct the coefficients—and extends the universal approximation property from continuous weight distributions to any dense-supp

Load-bearing premise

The proof's load-bearing premise is that a known universal approximation theorem for complex-valued activations can be invoked when the activation is real-to-complex and the input is allowed to be complex, even though the relevant feature expression is undefined for non-real inputs; a secondary unproven premise is that dense support forces every small ball to have positive sampling probability.

Editorial extensions

If this is right

Sparse, spike-and-slab weight sampling schemes gain a universal approximation guarantee with high probability, provided their support is dense in the weight space.
The high-probability guarantee becomes almost sure as the sample count grows, and the required m grows only logarithmically in 1/η.
Complex-valued activations (e.g., the complex exponentials used in random Fourier features) are covered by the theorem's statement, subject to the activation satisfying the non-polyharmonic condition.
The theorem is non-constructive: it certifies existence of approximating expansions and coefficient values but does not provide an algorithm to find them.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

As stated, the complex-input case likely does not go through: the proof invokes a universal approximation theorem that demands a complex-to-complex activation and real inputs, while the theorem is stated with φ:R→C and x∈C^n, leaving φ(ωᵀx+b) undefined for non-real inputs; a corrected theorem would probably restrict inputs to R^n or take φ:C→C.
The proof silently assumes that dense support in R^{n+1} implies every δ-ball has positive D-measure; this is true under the usual support definition but is not shown, and a distribution supported on a dense countable set with atomic masses would need an explicit justification that no ball has zero mass.
A testable next step is to specialize the result to random Fourier features on real inputs, where φ(t)=e^{it} (or a suitable complex extension) and the sampling distribution is dense; verifying the constants v* and q for this case would turn the logarithmic bound into a practical sample-size guide.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The one thing to know: the paper points at a real gap — universal approximation results for random basis expansions have not covered dense-support non-continuous sampling, which matters for spike-and-slab schemes — but the main theorem as written is ill-posed. φ:R→C is evaluated at ω_j^T x + b_j with x∈C^n; for non-real x the argument is not real, so the basis functions are undefined. The proof then invokes Voigtlaender's theorem, which requires φ:C→C and, as stated, approximates functions on R^n. That bridge does not hold.

What is genuinely new and good: the transfer idea — dense support means random samples can land near the fixed weights supplied by a universal approximation theorem, and a continuity lemma converts closeness in weights into closeness in functions. Lemma 2.4 is standard and correct for real inputs. For the real setting (f:R^n→R, φ:R→R), the strategy is sound and the high-probability claim should be recoverable after fixing the quantitative errors. The paper is honest that the result is non-constructive.

Soft spots beyond the domain error: the log bound in (12) is algebraically wrong, the inequality in (11) has the wrong direction, and the coefficient sum in (14) drops absolute values, so the error bound is not established even in the real case. The statement mixes 'non-polyharmonic' with a real-valued activation; for φ:R→C the term is undefined, and for φ:R→R it should read 'non-polynomial'. The complex activation case is simply not handled by the cited theorem.

This deserves a serious referee only because the real-input variant is plausibly repairable and the sampling question is relevant. As written, it should not go forward. If the authors restrict the theorem to real inputs, or replace φ:R→C with a genuinely complex activation and adjust the inputs accordingly, the resulting paper could be worth publishing.

Referee Report

4 major / 4 minor

Summary. The paper claims to prove a high-probability universal approximation theorem for random basis expansions whose weights are sampled from a distribution with dense support in R^{n+1}, allowing non-continuous sampling distributions and complex-valued activation functions. Theorem 3.1 states that for continuous f:C^n→C and a continuous non-polyharmonic φ:R→C, iid samples (ω_j,b_j) from such a distribution give, with high probability and with an explicit logarithmic width bound, an expansion Σ_j α_j φ(ω_j^T x + b_j) that uniformly approximates f on any compact K⊂C^n. The proof combines Pinkus's and Voigtlaender's universal approximation theorems with a continuity lemma and a union/intersection bound over finitely many sampled basis functions.

Significance. The underlying idea—that dense support of the sampling distribution, rather than absolute continuity, is enough to inherit universal approximation with high probability, together with an explicit width bound—is interesting and would be a useful contribution to the random basis expansion / random feature literature, especially for spike-and-slab type samplers. The manuscript is not constructive and relies on standard external UATs, but that is acceptable for an existence-style result. However, the central theorem as stated is mathematically ill-posed, the invocation of Voigtlaender's theorem is invalid, and the displayed sample-size bound is algebraically wrong. A correct version would likely require substantial reformulation, so the present manuscript is not suitable for publication.

major comments (4)

[Section 3, Theorem 3.1 (definition of φ_j)] The theorem is ill-posed for complex inputs. It takes f:C^n→C, K⊂C^n, but defines φ_j(x)=φ(ω_j^T x+b_j) with φ:R→C and (ω_j,b_j)∈R^{n+1}. For any x with nonzero imaginary part, ω_j^T x+b_j is not a real number, so φ_j(x) is undefined. Lemma 2.4, the only continuity mechanism, is restricted to K⊂R^n. Consequently, the object whose L∞ norm appears in (7) does not exist for general compact K⊂C^n.
[Section 2, Theorem 2.3 and Section 3, Eq. (8)] The application of Voigtlaender's theorem is invalid. Theorem 2.3 requires φ:C→C and, through Definition 2.1, approximates continuous functions on R^n, not on C^n. The proof's sentence 'If f:C^n→C and φ:R→C, this follows from Theorem 2.3' has no support in the stated theorem. Thus Eq. (8), which supplies the approximating q-term expansion, has no valid hypothesis for the complex-input case advertised in Theorem 3.1.
[Section 3, Eqs. (10)-(12)] The probability inequality has the wrong direction. Eq. (10) requires (1-(1-v_*)^l)^q ≥ 1-η. With m=ql, the l=m/q substitution gives (1-(1-v_*)^{m/q})^q ≥ 1-η, not ≤ as written in Eq. (11). Consequently Eq. (12) does not follow. In addition, the m-bound displayed in Theorem 3.1 contains an undefined constant C in the denominator and a log expression that is not derived from v_*; the statement's bound is therefore not a valid consequence of the argument.
[Section 3, proof of Theorem 3.1 (dense support and v_i>0)] The proof assumes that dense support implies v_i = P_D(B_{δ_i}(ω̃_i,b̃_i)) > 0 for every δ_i>0. This is true under the standard definition of support of a probability measure (every neighborhood of every support point has positive measure), but that definition is not given. If 'dense support' is interpreted only as the topological support being dense, the positivity of v_i is not automatic. This is a load-bearing point for the probabilistic statement and should be stated explicitly.

minor comments (4)

[Section 2, Lemma 2.4 proof] The proof refers to 'the compact set T⊂C bounded by M' but T is never defined; since φ:R→C, the relevant compact set should be a subset of R containing the values ω^T x+b. Also Eq. (6) has a typo: |ω_j^T x_1+b-(ω_j^T x_2+b)| should equal |ω_j^T(x_1-x_2)|, and the displayed line is missing a closing parenthesis and writes x for x_2.
[Section 3, proof of Theorem 3.1, Eq. (13) and following] Sums over the q approximating terms are indexed with n in several places (e.g., 'nX_{i=1}' and 'Let {j_i}_{i=1}^n'); these should be q. This is confusing because n is already the input dimension.
[Throughout] The phrase 'non-continuous weight distribution' is ambiguous; it could mean 'not continuous' in the topological sense or 'not absolutely continuous'. The paper should specify 'not having a density' or 'not absolutely continuous' where that is the intended meaning. Also, the abstract contains the typo 'jusify' and the introduction has 'the approximation' -> 'approximates' or similar.
[Section 3, Theorem 3.1 statement] The logarithmic bound as typeset is malformed: 'm≥log( 1/(1-(1-η)^{1/q}) q - 1 /(1-C) )' lacks parentheses and a log base, and the dependence on C versus v_* is not explained. Please rewrite with a clearly derived expression.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the dense-support high-probability covering argument is the actual derivation; the questionable application of Theorem 2.3 to C^n inputs is a correctness defect, not a circular reduction.

full rationale

The paper's central derivation is a standard high-probability covering argument, not a self-referential one. Theorem 3.1 assumes an approximating q-term expansion (8), which is imported from external universal approximation results: Pinkus (Theorem 2.2) for real activations and Voigtlaender (Theorem 2.3) for complex activations. Lemma 2.4 then converts closeness in weight space into closeness of the induced basis functions via continuity, and the dense-support condition on D ensures that each required δ-neighborhood has positive probability. Equations (9)-(12) are ordinary union/intersection probability estimates for independent samples; they do not reverse-engineer C or q from the conclusion. The coefficients are assigned by nearest-neighbor selection to the pre-existing q-term expansion, not fitted to the target beyond the external approximation theorem. There is no self-citation chain supporting the main claim, no parameter fitted and then renamed as a prediction, and no ansatz disguised as a derivation. The serious issue noted by the reader—that Theorem 2.3 as stated requires φ:C→C while Theorem 3.1 uses φ:R→C on complex inputs C^n—is a failure of the proof's hypotheses, not a circularity: the proof does not define its conclusion into existence, it simply relies on a theorem that may not cover the stated setting. Thus the circularity score is 0, with the caveat that mathematical correctness is a separate concern outside this pass.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The proof rests on external universal approximation theorems and standard analysis; no new entities or fitted parameters are introduced. The main assumption is dense support with positive ball probabilities.

assumptions (4)

standard math Pinkus theorem: a non-polynomial continuous activation φ:R→R yields universal approximation over real compact domains.
Used to guarantee existence of a q-term expansion for the real case in the proof of Theorem 3.1.
standard math Voigtlaender theorem: a locally bounded a.e.-continuous φ:C→C that is not almost polyharmonic yields universal approximation.
Invoked for the complex case, but misapplied because the paper's φ is R→C and inputs are asserted to be complex.
domain assumption If the support of a probability measure is dense in R^{n+1}, then every non-empty open ball has positive probability.
The proof requires v_i>0 for each δ_i-neighborhood; this follows from standard support definition but is not explicitly justified.
standard math Arzelà-Ascoli theorem
Used in Lemma 2.4 to show uniform convergence of feature functions.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On high probability of universal approximation in random basis expansions with non-continuous weight sampling." pith.science (2026). https://pith.science/paper/BEQCTYUU

@misc{pith2026260716551,
  author       = {Pith},
  title        = {Pith review of: On high probability of universal approximation in random basis expansions with non-continuous weight sampling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BEQCTYUU}},
  note         = {Machine review of arXiv:2607.16551}
}

read the original abstract

Random basis expansion (RBE) search the span of a randomly sampled basis to find the best approximation of a target function. They are equivalent to single layer neural networks where the hidden layer weights are chosen randomly. Universal approximation properties have been established for RBEs using continuous weight sampling distributions and real-valued activation functions. Our results extend the universal approximation property to RBEs that use non-continuous weight distributions with dense support in the weight space and that use complex-valued activation functions. The result shows such random bases have the universal approximation property with arbitrarily high probability.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 1 linked inside Pith

[1]

In: 46th Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL, USA, pp

Rahimi, A., Recht, B.: Uniform Approximation of Functions with Random Bases. In: 46th Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL, USA, pp. 555–561 (2008)

2008
[2]

Applied and Computational Harmonic Analysis62, 310–330 (2023)

Hashemi, A., Schaeffer, H., Shi, R., Topcu, U., Tran, G., Ward, R.: Generalization bounds for sparse random feature expansions. Applied and Computational Harmonic Analysis62, 310–330 (2023)

2023
[3]

Complex Systems2, 321–355 (1988)

Broomhead, D.S., Lowe, D.: Multivariable functional interpolation and adaptive networks. Complex Systems2, 321–355 (1988)

1988
[4]

Neurocomputing6(2), 163–180 (1994)

Pao, Y.-H., Park, G.-H., Sobajic, D.J.: Learning and generalization characteristics of the random vector functional-link net. Neurocomputing6(2), 163–180 (1994)

1994
[5]

IEEE Transactions on Neural Networks17(4), 879 (2006)

Huang, G.-B., Chen, L., Siew, C.-K.: Universal approximation using incremental constructive feed- forward networks with random hidden nodes. IEEE Transactions on Neural Networks17(4), 879 (2006)

2006
[6]

In: Proceedings., 11th IAPR International Conference on Pattern Recognition

Schmidt, W.F., Kraaijveld, M.A., Duin, R.P.W.: Feedforward neural networks with random weights. In: Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems, The Hague, NL, pp. 1–4 (1992)

1992
[7]

IEEE Transactions on Neural Networks6(6), 1320–1329 (1995)

Igelnik, B., Pao, Y.-H.: Stochastic choice of basis functions in adaptive function approximation and the functional-link net. IEEE Transactions on Neural Networks6(6), 1320–1329 (1995)

1995
[8]

In: Platt, J., Koller, D., Singer, Y., Roweis, S

Rahimi, A., Recht, B.: Random features for large-scale kernel machines. In: Platt, J., Koller, D., Singer, Y., Roweis, S. (eds.) NIPS’07: Proceedings of the 21st International Conference on Neural Information Processing Systems, vol. 20, pp. 1177–1184. Curran Associates, Inc., Vancouver, BC, CA (2007)

2007

Show all 16 references

[9]

Frontiers in Applied Mathematics and Statistics 10, 1284706 (2024)

Needell, D., Nelson, A.A., Saab, R., Salanevich, P., Schavemaker, O.: Random vector functional link networks for function approximation on manifolds. Frontiers in Applied Mathematics and Statistics 10, 1284706 (2024)

2024
[10]

Preprint at http://arxiv.org/abs/1810.04374

Sun, Y., Gilbert, A., Tewari, A.: On the pproximation Properties of Random ReLU Features. Preprint at http://arxiv.org/abs/1810.04374. (2019)

2019 arXiv
[11]

Sampling Theory, Signal Processing, and Data Analysis21(2), 27 (2023)

Saha, E., Schaeffer, H., Tran, G.: HARFE: hard-ridge random feature expansion. Sampling Theory, Signal Processing, and Data Analysis21(2), 27 (2023)

2023
[12]

Applied and Computa- tional Harmonic Analysis79, 101789 (2025)

Potts, D., Weidensager, L.: ANOV A-boosting for random Fourier features. Applied and Computa- tional Harmonic Analysis79, 101789 (2025)

2025
[13]

International Journal for Uncertainty Quantification14(4), 83–103 (2024)

Darges, J.E., Alexanderian, A., Gremaud, P.A.: Extreme learning machines for variance-based global sensitivity analysis. International Journal for Uncertainty Quantification14(4), 83–103 (2024)

2024
[14]

Neural Networks6(6), 861–867 (1993)

Leshno, M., Lin, V.Y., Pinkus, A., Schocken, S.: Multilayer feedforward networks with a non- polynomial activation function can approximate any function. Neural Networks6(6), 861–867 (1993)

1993
[15]

Acta Numerica8, 143–195 (1999)

Pinkus, A.: Approximation theory of the MLP model in neural networks. Acta Numerica8, 143–195 (1999)

1999
[16]

Applied and Computational Harmonic Analysis64, 33–61 (2023) 5

Voigtlaender, F.: The universal approximation theorem for complex-valued neural networks. Applied and Computational Harmonic Analysis64, 33–61 (2023) 5

2023

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [1]

In: 46th Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL, USA, pp

Rahimi, A., Recht, B.: Uniform Approximation of Functions with Random Bases. In: 46th Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL, USA, pp. 555–561 (2008)

2008

[2] [2]

Applied and Computational Harmonic Analysis62, 310–330 (2023)

Hashemi, A., Schaeffer, H., Shi, R., Topcu, U., Tran, G., Ward, R.: Generalization bounds for sparse random feature expansions. Applied and Computational Harmonic Analysis62, 310–330 (2023)

2023

[3] [3]

Complex Systems2, 321–355 (1988)

Broomhead, D.S., Lowe, D.: Multivariable functional interpolation and adaptive networks. Complex Systems2, 321–355 (1988)

1988

[4] [4]

Neurocomputing6(2), 163–180 (1994)

Pao, Y.-H., Park, G.-H., Sobajic, D.J.: Learning and generalization characteristics of the random vector functional-link net. Neurocomputing6(2), 163–180 (1994)

1994

[5] [5]

IEEE Transactions on Neural Networks17(4), 879 (2006)

Huang, G.-B., Chen, L., Siew, C.-K.: Universal approximation using incremental constructive feed- forward networks with random hidden nodes. IEEE Transactions on Neural Networks17(4), 879 (2006)

2006

[6] [6]

In: Proceedings., 11th IAPR International Conference on Pattern Recognition

Schmidt, W.F., Kraaijveld, M.A., Duin, R.P.W.: Feedforward neural networks with random weights. In: Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems, The Hague, NL, pp. 1–4 (1992)

1992

[7] [7]

IEEE Transactions on Neural Networks6(6), 1320–1329 (1995)

Igelnik, B., Pao, Y.-H.: Stochastic choice of basis functions in adaptive function approximation and the functional-link net. IEEE Transactions on Neural Networks6(6), 1320–1329 (1995)

1995

[8] [8]

In: Platt, J., Koller, D., Singer, Y., Roweis, S

Rahimi, A., Recht, B.: Random features for large-scale kernel machines. In: Platt, J., Koller, D., Singer, Y., Roweis, S. (eds.) NIPS’07: Proceedings of the 21st International Conference on Neural Information Processing Systems, vol. 20, pp. 1177–1184. Curran Associates, Inc., Vancouver, BC, CA (2007)

2007