REVIEW 4 major objections 4 minor 1 cited by
Recurrent Stochastic Configuration Networks with Incremental Blocks
T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Block-by-block reservoir growth preserves convergence while training faster.
desk verdict A reasonable block-increment extension of RSCN whose core approximation theorem does not cover the algorithm's acceptance rule, with a test-leaking validation design. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the block stochastic configuration algorithm: at each step a candidate subreservoir of N nodes is generated with random input weights, reservoir weights, and biases drawn from an adjustable uniform distribution, and it is accepted only if it passes the supervisory inequality in Eq. (12), which asks that the normalized correlation between the current residual and the subreservoir state exceed a threshold involving a contraction factor r and a decaying sequence µj+1. The accepted block that maximizes ξj+1 is added to a block-diagonal reservoir, whose weights are rescaled so that each subreservoir's maximum singular value is below one, guaranteeing the echo state property. The output weights are first set by global least squares and then updated online by the projection algorithm of Eq. (33). The proof mechanism for universal approximation is a residual contraction inequality: choosing output weights by least squares makes the new error satisfy ||e∗j+1||^2 ≤ (r + µj+1)||e∗j||^2, which forces the error to zero as j grows.
What would settle it
Run a single block-acceptance step on the Mackey-Glass data: generate many random subreservoir states, compute both the Eq. (12) quantity and the Eq. (24) projection norm of the residual onto the block state space, and check whether any state that passes Eq. (12) fails Eq. (24); if so, adding that block should produce a residual that does not contract by the claimed factor, contradicting Theorem 2 for the algorithm's actual selection rule.
Extended reading notes
Core claim
The paper claims that a recurrent stochastic configuration network with block increments, called a BRSCN, simultaneously adds multiple reservoir nodes in the form of subreservoirs, each configured by a supervisory mechanism, and still guarantees the two properties that make such networks useful for temporal data. Theorem 1 states that scaling each subreservoir's weight matrix so that its maximum singular value is below one preserves the echo state property. Theorem 2 states that if the subreservoir state satisfies the projection inequality in Eq. (24), then the training error satisfies lim j→∞ ||T − F_{j+1}|| = 0, giving universal approximation. Theorem 3 states that the online projection update of the output weights converges when the state signal satisfies a persistent excitation condition. The experimental sections report that across Mackey-Glass time series, nonlinear system identification, a debutanizer column soft sensor, and short-term power load forecasting, BRSCNs attain lower NMSE than baseline ESNs, growing ESNs, decoupled ESNs, and RSCNs while using smaller or comparable reservoirs and fewer iterations.
Load-bearing premise
The proof assumes that the scalar correlation inequality used to accept a subreservoir is strong enough to guarantee the projection-norm condition that drives the convergence theorem, and the paper does not prove that implication; the notation for block states is also ambiguous between vectors and matrices.
Editorial extensions
If this is right
- BRSCNs inherit the asymptotic training-error guarantee of RSCNs, so block-based construction does not sacrifice the theoretical basis for randomized reservoir design.
- The block-diagonal reservoir with singular-value scaling preserves the echo state property, meaning the network's temporal behavior remains stable for long input sequences.
- The online projection update with persistent excitation gives a convergence guarantee for output weights, supporting adaptive use in changing industrial processes.
- Larger subreservoir sizes, such as Nsub = 10, reduce the number of construction iterations and yield the smallest reported NMSE across the tested benchmarks.
- The reported results suggest that block increments produce more compact reservoirs, with testing NMSE that is often lower than point-by-point RSCN growth at similar or smaller reservoir sizes.
Reading between the lines
- A possible gap is that the algorithm accepts blocks using the scalar correlation inequality in Eq. (12), while Theorem 2 assumes the projection-norm inequality in Eq. (24); the paper does not prove these are equivalent, so the universal approximation guarantee might only hold for a subset of the blocks the algorithm actually selects.
- If the implication between Eq. (12) and Eq. (24) holds, block increments could be extended beyond RSCNs to other stochastic-configuration architectures, such as deep or multi-scale reservoirs, without re-deriving the contraction argument.
- The persistent excitation condition could be interpreted as a requirement on the spectral richness of the reservoir state, which would give practitioners a quantifiable diagnostic for when online weight updates are reliable.
- A direct numerical test of the Eq. (12)-to-Eq. (24) implication on simple Mackey-Glass data, computing both quantities for randomly generated subreservoirs, would settle whether the theoretical guarantee applies to the actual construction procedure.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes block recurrent stochastic configuration networks (BRSCNs), which add subreservoirs of N reservoir nodes at a time. The construction is guided by a supervisory inequality (Eq. (12)) for accepting candidate blocks, a spectral-radius scaling rule (Eq. (13)) is intended to enforce the echo state property, and the output weights are updated online with a projection algorithm. Theorem 1 states a condition for the echo state property, Theorem 2 claims universal approximation under the inequality in Eq. (24), and Theorem 3 claims parameter convergence under persistent excitation. Experiments compare BRSCN with ESN, DESN, GESN, and RSCN on Mackey-Glass prediction, nonlinear system identification, debutanizer soft sensing, and short-term power-load forecasting.
Significance. If the three theorems were fully established, BRSCN would be a useful block-incremental extension of RSCN: block construction would reduce the number of iterations while preserving the universal approximation argument, and the online projection update would carry a convergence guarantee. The experimental comparison is broad, includes several block sizes, and reports compact reservoirs with competitive NRMSE on four tasks. The paper does not provide code or machine-checked proofs, and the main theory is an adaptation of the earlier RSCN proof [26] and the block-SCN framework [28], so the incremental contribution is moderate. Nevertheless, the extension is natural and the experiments suggest practical value; most of the technical gaps are local and appear repairable.
major comments (4)
- [Sections III-A and III-C, Eqs. (12), (14), (24)] The connection between Algorithm 1's acceptance rule and Theorem 2's hypothesis is missing. In Eq. (12), X^{(j+1),i} is an N x n_max matrix while e_{j,q} is a 1 x n_max row vector, so the expressions <e_{j,q}, X>^2 / <X, X> are undefined unless a convention is supplied, and none is stated. If one adopts <e,X> = eX^T and <X,X> = tr(XX^T), then Eq. (12) is equivalent to C_q >= delta*_q with C_q <= eX^T(XX^T)^{-1}Xe^T = P_q, so the test can be sufficient for the projection condition only after proving a bridging lemma; conversely, the test can reject blocks with P_q >= delta*_q when the block contains large-norm components orthogonal to e_{j,q}. In addition, Theorem 2 states Eq. (24) with W_out evaluated by Eq. (16), while the proof uses the new-block-only least-squares solution Eq. (22); these are different objects. Without repaired definitions and a bridging lemma, the universal approximation conclusion is not established for the blocks the algorithm actually accepts.
- [Section III-C, Theorem 2 and Algorithm 1] No argument shows that, at every residual stage, a candidate block satisfying Eq. (12) exists within the Gmax random draws. Algorithm 1's fallback of increasing r and repeating is not proved to terminate with a valid block. The density assumption on span(Gamma) in Theorem 2 is not used to prove such an existence statement for block states; in the scalar SCN/RSCN theory this is supplied by a construction or probabilistic density argument, and the block analogue is absent. Therefore, even if the inequality implication in the previous comment were repaired, the theorem would not yet guarantee that Algorithm 1 produces an infinite sequence of accepted blocks whose residuals converge to zero.
- [Section III-B, Remark 2, Eq. (13) and Theorem 1] Theorem 1 requires the strict scaling condition alpha < rho_max / sigma_max to ensure that the maximum singular value of each scaled subreservoir is below one, but Remark 2 and Eq. (13) specify only 0 < alpha < 1 in the scaling W <- (alpha / rho_max) W. The experimental setting alpha in [0.5, 1] (Section V) does not enforce the theorem's bound, so the echo state property guarantee as stated does not cover the models actually trained. The proof of Theorem 1 is correct under Eq. (17), but the condition needs to be incorporated into Remark 2 and Algorithm 1.
- [Section IV-B, Theorem 3] The persistent-excitation proof contains two load-bearing errors. First, Eq. (34) is a window-integral inequality, but the text immediately asserts the pointwise bound eta1 >= g(n)^T g(n) >= eta2, which does not follow from an integral bound. Second, Eq. (37) treats the parameter error update as (I - P gamma g^T g) E(n-1), but with g a column vector and E a row vector the update is E(n) = E(n-1)(I - gamma P g g^T), not the scalar form used; the subsequent Lyapunov difference in Eqs. (39)-(40) is therefore dimensionally inconsistent. As written, the convergence claim of Theorem 3 is not proven.
minor comments (4)
- [Section III-C, Theorem 2] The symbol Gamma in 'span(Gamma)' is never defined, and the superscript asterisk on e^*_j is not explained; the reader must infer that it denotes the residual after the optimal output weights are used.
- [Section V.A, Fig. 3 and Table I] The text after Fig. 3 states that the optimal BRSCN reservoir size for the MG task is 60, while Table I reports a reservoir size of 50 for the same task; please reconcile this discrepancy.
- [Algorithm 1] Algorithm 1 is titled 'DeepRSC', although the paper introduces BRSCN; if this title is not intentional, it should be changed to match the proposed method.
- [Section IV-B, Eq. (34)] Equation (34) is written as an integral over [n0, n0+nw] in a setting where the projection update is discrete-time; please clarify whether the integral denotes a summation and define n0 and nw.
Circularity Check
No significant circularity: Theorem 2 is a sufficient-condition proof, and the gap connecting Eq. (12) to Eq. (24) is a correctness issue, not a self-referential reduction.
full rationale
The paper's central universal approximation claim does not reduce to its own inputs by construction. Theorem 2 assumes the projection inequality (24), namely <e*_j, W_out X> >= delta*, and then derives the contraction ||e*_{j+1}||^2 <= (r + mu) ||e*_j||^2. This is a genuine sufficient-condition argument: the condition is a data-dependent inequality on the residual and the new block state, and the conclusion follows from it mathematically. It is not a fitted parameter renamed as a prediction. Algorithm 1, however, uses the scalar correlation test in Eq. (12)/(14), which is not the same condition as Eq. (24). The missing lemma relating these two inequalities, together with the ambiguous matrix/vector convention for X in Eqs. (12)-(14), is an omitted derivation step and a potential correctness flaw. But it is not circularity: there is no definition that makes Eq. (12) equal Eq. (24) by construction, and neither condition is defined in terms of the theorem's conclusion. Theorem 1 is a straightforward application of the standard singular-value sufficient condition for the echo state property, citing Jaeger's external ESP result; it does not depend on the authors' own prior work. Theorem 3's persistent-excitation assumption is a condition on the regressor g(n) used in a Lyapunov convergence argument, not a condition fitted from the predicted target. Self-citations to [26] provide background and a stability remark, but the block-incremental universal approximation proof is an extension of [28] and is otherwise self-contained. No load-bearing claim in the paper is forced by self-citation or by definition, so the circularity score should be 0.
Assumptions & free parameters
free parameters (4)
- Subreservoir size Nsub =
10 (grid search)
- Reservoir scaling factor alpha =
0.5 to 1
- Contractive sequence r and mu =
r in {0.9, ..., 0.99999}; mu_{j+1} = (1-r)/((j+1)*N)
- Projection algorithm constants gamma and c =
gamma > 0, small c
assumptions (5)
- standard math span(Gamma) is dense in L2
- standard math Echo state property follows from maximum singular value of the reservoir weight matrix being less than 1
- domain assumption Input signals satisfy the persistent excitation condition in Eq. (34) as a window-integral bound
- ad hoc to paper The scalar acceptance inequality Eq. (12) implies the projection inequality Eq. (24) for multi-node subreservoirs
- ad hoc to paper The subreservoir state X in Eqs. (12)-(14) has a well-defined vector/matrix inner product structure
Cite this review
Pith. "Pith review of Recurrent Stochastic Configuration Networks with Incremental Blocks." pith.science (2026). https://pith.science/paper/KJAAFK7F
@misc{pith2026241111303,
author = {Pith},
title = {Pith review of: Recurrent Stochastic Configuration Networks with Incremental Blocks},
year = {2026},
howpublished = {\url{https://pith.science/paper/KJAAFK7F}},
note = {Machine review of arXiv:2411.11303}
}
read the original abstract
Recurrent stochastic configuration networks (RSCNs) have shown promise in modelling nonlinear dynamic systems with order uncertainty due to their advantages of easy implementation, less human intervention, and strong approximation capability. This paper develops the original RSCNs with block increments, termed block RSCNs (BRSCNs), to further enhance the learning capacity and efficiency of the network. BRSCNs can simultaneously add multiple reservoir nodes (subreservoirs) during the construction. Each subreservoir is configured with a unique structure in the light of a supervisory mechanism, ensuring the universal approximation property. The reservoir feedback matrix is appropriately scaled to guarantee the echo state property of the network. Furthermore, the output weights are updated online using a projection algorithm, and the persistent excitation conditions that facilitate parameter convergence are also established. Numerical results over a time series prediction, a nonlinear system identification task, and two industrial data predictive analyses demonstrate that the proposed BRSCN performs favourably in terms of modelling efficiency, learning, and generalization performance, highlighting their significant potential for coping with complex dynamics.
Figures
Figures from the paper (7 more)
Forward citations
Cited by 1 Pith paper
-
Recurrent Stochastic Configuration Networks with Hybrid Regularization for Nonlinear Dynamics Modelling
A hybrid LASSO plus L2-regularized RSCN is proposed for nonlinear dynamics modelling, with improved reported test accuracy but a gap in the universal approximation proof.
Reference graph
Works this paper leans on
-
[26]
Recurrent stochastic configuration networks for temporal data analytics,
D. Wang and G. Dang, “Recurrent stochastic configuration networks for temporal data analytics,” arXiv: 2406.16959v2, Sep. 2024
arXiv 2024
-
[28]
Stochastic configuration networks with block increments for data modeling in process industries,
W. Dai, D. Li, P. Zhou, and T. Chai, “Stochastic configuration networks with block increments for data modeling in process industries,” Inf. Sci., vol. 484, pp. 367-386, May 2019
work page 2019
-
[1]
A compre- hensive review for industrial applicability of artificial neural networks,
M. R. G. Meireles, P. E. M. Almeida, and M. G. Simoes, “A compre- hensive review for industrial applicability of artificial neural networks,” IEEE Trans. Ind. Electron., vol. 50, no. 3, pp. 585-601, Jun. 2003. 12
work page 2003
-
[2]
A deep dual adversarial network for cross-domain recommendation,
Q. Zhang, W. Liao, G. Zhang, B. Yuan, and J. Lu, “A deep dual adversarial network for cross-domain recommendation,” IEEE Trans. Knowl. Data Eng., vol. 35, no. 4, pp. 3266-3278, Apr. 2023
work page 2023
-
[3]
Multi-scale dynamic convolu- tional network for knowledge graph embedding,
Z. Zhang, Z. Li, H. Liu, and N. Xiong, “Multi-scale dynamic convolu- tional network for knowledge graph embedding,” IEEE Trans. Knowl. Data Eng., vol. 34, no. 5, pp. 2335-2347, May 2022
work page 2022
-
[4]
R. Pascanu, T. Mikolov, and Y . Bengio, “On the difficulty of training recurrent neural networks, in International conference on machine learn- ing,” in Proceedings of the 30th International Conference on Machine Learning, PMLR, vol. 28, no. 3, pp. 1310-1318, 2013
work page 2013
-
[5]
Deep reservoir computing: A critical experimental analysis,
C. Gallicchio, A. Micheli, and L. Pedrelli, “Deep reservoir computing: A critical experimental analysis,” Neurocomputing, vol. 268, pp. 87–99, Dec. 2017
work page 2017
-
[6]
Periodic weather-aware LSTM with event mechanism for parking behavior pre- diction,
F. Zhang, Y . Liu, N. Feng, C. Yang, J. Zhai, and S. Zhang, “Periodic weather-aware LSTM with event mechanism for parking behavior pre- diction,” IEEE Trans. Knowl. Data Eng., vol. 34, no. 12, pp. 5896-5909, Dec. 2022
work page 2022
Show all 30 references
-
[7]
Survey: reservoir computing ap- proaches to recurrent neural network training,
M. Luko ˇseviˇcius and H. Jaeger, “Survey: reservoir computing ap- proaches to recurrent neural network training,” Computing Science Review, vol. 3, no. 3, pp. 127–149, Aug. 2009
2009
-
[8]
Randomness in neural networks: an overview,
S. Scardapane and D. Wang, “Randomness in neural networks: an overview,” Wiley Interdiscip. Rev.: Data Min. Knowl. Discovery, vol. 7, no. e1200, Feb. 2017
2017
-
[9]
The echo state approach to analysing and training recurrent neural networks-with an erratum note,
H. Jaeger, “The echo state approach to analysing and training recurrent neural networks-with an erratum note,” German Nat. Res. Center Inf. Technol., Bonn, Germany, Tech. Rep. GMD, 148, 2001
2001
-
[10]
Real-time computing without stable states: a new framework for neural computation based on perturbations,
W. Maass, T. Natschl ¨ager, and H. Markram, “Real-time computing without stable states: a new framework for neural computation based on perturbations,” Neural Comput., vol. 14, no. 11, pp. 2531-2560, Nov. 2002
2002
-
[11]
Evolving dual-threshold bienenstock-cooper-munro learning rules in echo state networks,
X. Wang, Y . Jin, W. Du, and J. Wang, “Evolving dual-threshold bienenstock-cooper-munro learning rules in echo state networks,” IEEE Trans. Neural Networks Learn. Syst., vol. 35, no. 2, pp. 1572-1583, Feb. 2024
2024
-
[12]
A systematic review of echo state networks from design to application,
C. Sun, M. Song, D. Cai, B. Zhang, S. Hong, and H. Li, “A systematic review of echo state networks from design to application,” IEEE Trans. Artif. Intell., vol. 5, no. 1, pp. 23-37, Jan. 2024
2024
-
[13]
Deep fuzzy echo state networks for machinery fault diagnosis,
S. Zhang, Z. Sun, M. Wang, J. Long, Y . Bai, and C. Li, “Deep fuzzy echo state networks for machinery fault diagnosis,” IEEE Trans. Fuzzy Syst., vol. 28, no. 7, pp. 1205-1218, Jul. 2020
2020
-
[14]
Minimum complexity echo state network,
A. Rodan and P. Tino, “Minimum complexity echo state network,” IEEE Trans. Neural Networks, vol. 22, no. 1, pp. 131-144, Jan. 2011
2011
-
[15]
Optimization and applications of echo state networks with leaky-integrator neurons,
H. Jaeger, M. Luko ˇseviˇcius, D. Popovici, and U. Siewert, “Optimization and applications of echo state networks with leaky-integrator neurons,” Neural Networks, vol. 20, no. 3, pp. 35-352, Apr. 2007
2007
-
[16]
Echo state property of deep reservoir computing networks,
C. Gallicchio and A. Micheli, “Echo state property of deep reservoir computing networks,” Cognit. Comput., vol. 9, no. 3, pp. 337–350, May 2017
2017
-
[17]
Deep echo state network: a brief survey,
C. Gallicchio and A. Micheli, “Deep echo state network: a brief survey,” arXiv:1712.04323v4, Sep. 2020
2020 arXiv
-
[18]
Pruning and regular- ization in reservoir computing,
X. Dutoit, B. Schrauwen, and J. Campenhout, “Pruning and regular- ization in reservoir computing,” Neurocomputing, vol. 72, no. 7-9, pp. 1534-1546, Mar. 2009
2009
-
[19]
Growing echo-state network with multiple subreservoirs,
J. Qiao, F. Li, H. Han, and W. Li, “Growing echo-state network with multiple subreservoirs,” IEEE Trans. Neural Networks Learn. Syst., vol. 28, no. 2, pp. 391-404, Feb. 2017
2017
-
[20]
Echo state networks: novel reservoir selection and hyperparameter optimization model for time series forecasting,
C. H. Valencia, M. Vellasco, and K. Figueiredo, “Echo state networks: novel reservoir selection and hyperparameter optimization model for time series forecasting,” Neurocomputing, vol. 545, no. 126317, Aug. 2023
2023
-
[21]
PSO-based growing echo state network,
Y . Li and F. Li, “PSO-based growing echo state network,” Appl. Soft Comput., vol. 85, no. 105774, Dec. 2019
2019
-
[22]
Parameterizing echo state networks for multi-step time series prediction,
J. Viehweg, K. Worthmann, and P. M ¨ader, “Parameterizing echo state networks for multi-step time series prediction,” Neurocomputing, vol. 522, pp. 214-228, Feb. 2023
2023
-
[23]
Editorial: randomized algorithms for training neural net- works,
D. Wang, “Editorial: randomized algorithms for training neural net- works,” Inf. Sci., vol. 364–365, pp. 126–128, Oct. 2016
2016
-
[24]
Insights into randomized algorithms for neural networks: practical issues and common pitfalls,
M. Li and D. Wang, “Insights into randomized algorithms for neural networks: practical issues and common pitfalls,” Inf. Sci., vol. 382–383, pp. 170–178, Mar. 2017
2017
-
[25]
Stochastic configuration networks: Fundamentals and algorithms,
D. Wang and M. Li, “Stochastic configuration networks: Fundamentals and algorithms,” IEEE Trans. Cybern., vol. 47, no. 10, pp. 3466-3479, Oct. 2017
2017
-
[27]
Adaptive filtering prediction and control,
G. Goodwin and K. Sin, “Adaptive filtering prediction and control,” Courier Corporation, 2014
2014
-
[29]
Decoupled echo state networks with lateral inhibition,
Y . Xue, L. Yang, and S. Haykin, “Decoupled echo state networks with lateral inhibition,” Neural Networks, vol. 20, no. 3, pp. 365-376, Apr. 2007
2007
-
[30]
Soft sensors for product quality monitoring in debutanizer distillation columns,
L. Fortuna, S. Graziani, and M. G. Xibilia, “Soft sensors for product quality monitoring in debutanizer distillation columns,” Control Eng. Pract., vol. 13, no. 4, pp. 499-508, Apr. 2005
2005
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.