{"id":"670348d3-eeb6-40c7-953a-5c16ca92dd89","arxiv_id":"2412.00070","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A hybrid LASSO plus L2-regularized RSCN is proposed for nonlinear dynamics modelling, with improved reported test accuracy but a gap in the universal approximation proof.","lead":"This paper combines LASSO order-variable selection with an L2-regularized recurrent stochastic configuration network (RSCN) that models the residuals of the linear LASSO model, targeting nonlinear dynamic systems and industrial time-series prediction. The authors report lower test error than several baselines, but the universal approximation proof contains a flawed step and the experiments are not fully controlled.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1's key inference after Eq. (18) is false: a global ridge solution can have larger unregularized error than the constructive one-node update, so the universal approximation claim for the actual algorithm is unsupported.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing step, and I agree. The counterexample shows that the inference after Eq. (18) is not merely unproved but false under the theorem's own hypotheses, so the paper's central theoretical contribution does not cover the actual algorithm, which uses global regularized least squares. The empirical results could still be useful, but they do not repair the theorem. The theoretical claim needs either a different proof, for example proving monotone decrease of the actual ridge residual or using constructive weights throughout, or a revised statement. The experimental lag asymmetry is secondary to this theoretical gap. Since this supports the reader's REJECT verdict, no change to the verdict is recommended.","tokens_in":16333,"tokens_out":11192,"duration_ms":104639,"concrete_test":"Implement the one-sample counterexample above and compare the two squared errors; if the global ridge residual exceeds the constructive residual, the theorem's inference is false. Then run a small version of Algorithm 1 on a system where the current reservoir already fits training data well and a nearly collinear node is added, switching between constructive weights (15) and global ridge weights (17)-(18); if training error jumps upward when global weights are used, the current proof cannot guarantee the claimed convergence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The theorem's pivotal step is the claim after Eq. (18) that the global regularized least-squares residual satisfies ||tilde e_{N+1}||^2 <= ||e_{N+1}||^2. This does not follow from minimizing (17), which includes C||Wout||^2; a solution with smaller regularized objective can have larger unregularized error. A one-sample counterexample satisfies the theorem's hypotheses. Take previous feature x=[1], target t=[1], C=0.1, and new node g=[2] with b_g=2 and r=0.9, mu=0. The previous ridge weight is w_1=1/1.1≈0.909, so e_1≈0.0909. Inequality (14) holds, since <e_1,g>^2≈0.0331 exceeds ((4.1)^2/4.2)*0.1*||e_1||^2≈0.0033. The constructive update (15) gives alpha≈0.0443, residual e_2≈0.0022, squared error≈4.9e-6. The global ridge solution from (18) is (v,w)≈(0.196,0.392), residual≈0.0196, squared error≈3.8e-4, roughly 80 times larger. Thus the inferred inequality is false. Since Algorithm 1 uses the global solution (17)-(18) at every step, Theorem 1 does not establish convergence of the proposed LASSO-RSCN-L2; the universal approximation claim is unsupported. The experimental comparison also gives LASSO variants extra lagged inputs, but the broken theorem is the primary blocker.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a hybrid recurrent stochastic configuration network (LASSO-RSCN-L2) for nonlinear dynamics modelling. It first uses LASSO to select lagged order variables, then incrementally builds an RSCN whose output weights are obtained by an L2-regularized least-squares solve, and finally applies an online projection update for the output weights. The central theoretical claim is a universal approximation theorem (Theorem 1) asserting that the regularized incremental construction drives the training residual to zero. The experimental section reports NRMSE comparisons on a nonlinear system identification benchmark and two industrial datasets (debutanizer soft sensing and short-term power load forecasting), claiming that the proposed model achieves the lowest testing NRMSE.","tokens_in":16724,"tokens_out":7527,"duration_ms":66260,"significance":"The hybrid order-selection plus residual-compensation idea is practically motivated, and the paper gives a complete algorithmic description with mean/variance statistics over 50 trials, which facilitates reimplementation. If the universal approximation guarantee were valid, the regularized supervisory node-selection mechanism would be a useful extension of stochastic configuration network theory. However, the proof of Theorem 1 contains a false inference that is load-bearing for the main theoretical claim, and the experimental comparisons use different input-feature sets for the LASSO variants than for the baselines. As a result, neither the theoretical nor the empirical headline claims are established as stated.","major_comments":[{"comment":"The assertion \"It is easily inferred that ||tilde e_{N+1}||^2 <= ||e_{N+1}||^2\" is false. The global solution of (17)-(18) minimizes the regularized objective, not the unregularized squared error, so a minimizer of the regularized problem can have larger unregularized residual than the constructive update (15) analyzed in the proof. For example, with one training sample (x,t)=(1,1), C=0.1, previous ridge weight 1/1.1, and new node g=2, inequality (14) holds with r=0.9, mu=0, and b_g=2; the constructive update gives residual squared about 4.9e-6, while the global ridge solution (18) gives residual squared about 3.8e-4. Therefore inequality (19) does not follow, and Theorem 1 does not establish the claimed limit for Algorithm 1, which uses the global ridge solution at every step.","section":"Section III-B, proof of Theorem 1, text after Eq. (18)"},{"comment":"There is a mismatch between the object analyzed in Theorem 1 and the object used in the proposed algorithm. The theorem proves a contraction property for the constructive single-node output weight (15), but Algorithm 1 (line 29 and Step 7 of the textual description) recomputes all output weights by the global regularized least-squares formula (17)-(18). The node-selection criterion (25) is also derived from the constructive update. Consequently, even if inequality (14) were sufficient for the constructive construction, it does not control the residual of the global solution, so the convergence proof does not apply to the algorithm as stated.","section":"Section III-B, Eqs. (14)-(18) and Algorithm 1"},{"comment":"The baseline comparison is not matched in input features. In the nonlinear system identification task, ESN, RSCN, and their variants predict y(n+1) from [y(n), u(n)], while the LASSO-based frameworks are given ten-order delays for the inputs. In the debutanizer case, non-LASSO models use [u1(n),...,u5(n), y(n-1)], whereas the LASSO variants implement second-order delays for every input and then select variables such as u1(n-2) and u4(n-2) from that larger set. The reported improvements of LASSO-RSCN-L2 over RSCN-L2 may therefore be due to the additional lagged inputs rather than to the hybrid regularization or the RSCN architecture, so the empirical claim that the proposed method outperforms other models is not cleanly supported.","section":"Section IV-A and IV-B, experimental setup"},{"comment":"The paper claims that the echo state property is 'naturally inherited' and that convergence of the projection updates is guaranteed, but for both statements the proof is delegated to the authors' unpublished preprint [22]. Since [22] is not part of the peer-reviewed record, these guarantees cannot be independently verified from the manuscript, and the reliance on an unpublished source for a stated contribution weakens the support for the stability and convergence claims.","section":"Contributions (item 3) and Remarks 1 and 3"}],"minor_comments":[{"comment":"The line 'Set mu_{N+1} = (1-r)/N + 1;' is ambiguous; with standard operator precedence it means ((1-r)/N)+1, which tends to 1 and would violate the assumption lim mu_{N+1}=0 and mu_{N+1} <= (1-r). If the intended value is (1-r)/(N+1), it should be written with brackets.","section":"Algorithm 1, line 11"},{"comment":"Reference [23] is a duplicate of reference [5]; they list the same paper by Kamalapurkar, Fischer, Obuz, and Dixon with identical title, journal, volume, pages, and year.","section":"References"},{"comment":"The inner product notation <e_{N,q}, g_{N+1}> is used before it is formally introduced; the authors should state explicitly that these are Euclidean inner products over the training samples.","section":"Eqs. (8) and (9)"},{"comment":"The summation in (27) contains garbled notation ('dK_B d' and 'pdk_B d'); the index ranges and the meaning of the coefficient should be written out clearly.","section":"Eq. (27)"},{"comment":"Table I lists ranges for reservoir size, sparsity, and scaling factor, but not for the regularization coefficient C, even though the text states that grid search is used for network parameters; Figs. 4 and 11 also omit axis labels for the vertical axis, making the reported optimal combinations hard to read.","section":"Table I and Figs. 4 and 11"}],"recommendation":"reject","confidential_remarks":"The manuscript relies on two unpublished preprints by the same authors for core guarantees ([22] for echo state property and projection convergence, [30] for the block extension), which makes independent verification difficult. More importantly, the false inference in Theorem 1 affects the central theoretical contribution, and the experimental baselines differ in input-feature sets, so the reported empirical gains are not attributable to the proposed method alone. These issues cannot be fixed by local edits within the current scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the proposed model is a plausible engineering combination, but the central theorem has a real hole. The step after Eq. (18) — \"It is easily inferred that ||\\tilde e_{N+1}||^2 ≤ ||e_{N+1}||^2\" — is false for regularized least squares. Ridge regression minimizes a penalized objective; it can return weights whose unregularized training error is larger than the constructive single-node update. The counterexample in the stress-test note (one sample, x=1, t=1, C=0.1, g=2) checks out: inequality (14) holds, the constructive update gives squared error about 5e-6, and the global ridge solution gives about 3.8e-4. Since Algorithm 1 uses the global solution, Theorem 1 does not prove convergence of the actual algorithm. This is not a minor typo; it is the load-bearing step.\n\nWhat is genuinely new: LASSO order selection feeding residuals into an L2-regularized RSCN, plus the modified supervisory inequality (14), is a straightforward but real extension of the authors' own RSCN framework. The experiments are clearly reported with means and standard deviations, and the authors are upfront in the conclusion that no theoretical basis for choosing C is provided. The writing is readable and the industrial tasks are relevant.\n\nSoft spots beyond the theorem: the baselines do not receive the same input information. In the system identification experiment, ESN and RSCN get only [y(n), u(n)] while LASSO variants are allowed ten lags. That alone could account for part of the reported improvement. The LASSO coefficient selection rule — maximize the sum of absolute coefficients across orders — is unusual and not justified; standard cross-validated LASSO would be easier to defend. The echo state property and projection-algorithm convergence are cited to an unpublished preprint [22], so the paper is not self-contained, though that alone is not a fatal flaw.\n\nWho this is for: readers working on randomized recurrent networks and industrial soft sensors. They should treat the experiments as suggestive and the universal approximation claim as currently unsupported.\n\nRecommendation: this deserves a serious referee, not a desk reject. The error is concrete and fixable in principle — correct the proof, weaken the claim, or drop the theorem to a remark and resubmit as an empirical paper. As it stands, the central theoretical contribution is not proven.","headline":"The hybrid LASSO+RSCN idea is a reasonable engineering combination, but the universal approximation theorem has a load-bearing gap: the global ridge solution is not guaranteed to have smaller unregularized error than the constructive update.","tokens_in":17208,"tokens_out":3167,"would_cite":false,"duration_ms":29095,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two-stage regularization — LASSO for delays, L2 for the residual learner — is claimed to give recurrent stochastic configuration networks the universal approximation property and the best test error on three nonlinear dynamics tasks.","keywords":["recurrent stochastic configuration networks","LASSO order selection","L2 regularization","universal approximation","nonlinear system identification","soft sensing","power load forecasting","echo state property"],"falsifier":"Construct a case with two training samples and one existing reservoir node whose current output weight already makes the residual exactly zero, then add one candidate node and compute both $\\|\\tilde{e}_{N+1}\\|^2$ from the global ridge solution (17)-(18) and $\\|e_{N+1}\\|^2$ from the single-node update (15); the proof needs $\\|\\tilde{e}_{N+1}\\|^2 \\le \\|e_{N+1}\\|^2$, and a simple numerical example will show whether that inequality holds for any $C>0$.","tokens_in":7,"feed_emoji":"📈","tokens_out":7414,"duration_ms":129538,"temperature":0.7,"pith_summary":"This paper tries to establish that a two-stage regularized recurrent stochastic configuration network can model nonlinear dynamic systems with unknown or time-varying delays. The first stage uses LASSO to pick the important lagged inputs; the second stage trains an L2-regularized RSCN on the residuals left by the linear model. The authors prove a universal approximation theorem for this construction and report the lowest test error on a nonlinear identification task and two industrial forecasting tasks. A sympathetic reader would care because the approach offers a randomized, non-backpropagation route to recurrent modeling with a stated convergence guarantee.","feed_headline":"Hybrid LASSO-L2 RSCN posts lowest test errors on three tasks","feed_subtitle":"LASSO finds the key delays; an L2-regularized recurrent SCN fits the residuals and earns the best test NRMSE.","key_machinery":"The hybrid model's machinery is a two-stage cascade: LASSO (11) selects the delay variables, and the residuals $\\hat{Y}=T-Y_{\\mathrm{LASSO}}$ are learned by an incrementally built reservoir whose new node must satisfy inequality (14) with the L2-shifted constants $(b_g^2+C)^2/(b_g^2+2C)$, and whose output weight is set by (15), so that Theorem 1's contraction bound $\\|e_{N+1}\\|^2 \\le (r+\\mu_{N+1})\\|e_N\\|^2$ delivers the universal approximation result. The echo state property is inherited from the lower-triangular reservoir matrix (5)-(6), giving the network a stable dynamic response to small input variations.","core_discovery":"On its own terms, the central claim is that the hybrid LASSO-RSCN-L2 model — LASSO for delay-order selection, then a recurrent stochastic configuration network with an L2 penalty fit to the residual target — has the universal approximation property and achieves the best testing NRMSE among the compared models on all three datasets. Theorem 1 states that if each added node satisfies inequality (14) and its output weight is set by (15), then the residual norm obeys $\\|e_{N+1}\\|^2 \\le (r+\\mu_{N+1})\\|e_N\\|^2$ with $\\mu_{N+1} \\to 0$, so $\\lim_{N \\to \\infty}\\|e_N\\| = 0$. The paper further claims that the L2 penalty plus a projection-based online update keeps the output weights bounded and lets the network adapt quickly to new samples, while the triangular reservoir matrix preserves the echo state property.","pith_inferences":["If the flagged proof gap is real, the universal-approximation theorem as stated may not be established, yet the empirical gains could still hold through the early-stopping and regularization mechanism rather than through the specific contraction inequality.","The two-stage residual idea could be transplanted to other reservoir models, such as leaky-integrator echo state networks, to test whether the LASSO-plus-residual compensation, not the RSCN construction itself, is what drives the reported accuracy.","A natural extension is to replace plain LASSO with group-sparse or adaptive LASSO so that groups of lagged inputs are selected together, which could improve stability on correlated industrial variables.","A direct ablation — LASSO-RSCN-L2 versus a single-stage RSCN with both L1 and L2 penalties on the same tasks — would isolate how much of the gain comes from the LASSO pre-filter rather than from the L2 regularizer."],"forward_implications":["If the theorem holds, the construction yields a reservoir whose training error provably goes to zero without backpropagation, as long as each newly added node passes the supervisory inequality (14).","The LASSO pre-filter gives an interpretable selection of which lagged inputs matter, and on the reported tasks the chosen delays align with the governing dynamics, while the reservoir size shrinks compared to plain ESN and RSCN baselines.","The L2 penalty combined with the projection-based update keeps the output weights regularized and adaptable, which the paper argues mitigates the ill-posedness of temporal data with unknown orders.","The empirical claim, if correct, is that LASSO-RSCN-L2 posts the lowest testing NRMSE across all three tasks, with notably fewer reservoir nodes than the non-LASSO alternatives.","The two-stage residual-compensation idea is presented as a general strategy: first capture the linear lag structure, then let the recurrent randomized learner model what the linear part misses."],"supporting_citations":[{"why":"Introduces the stochastic configuration network and the supervisory inequality for random parameter assignment that the RSCN builds on.","marker":"[18]"},{"why":"Defines recurrent stochastic configuration networks, the triangular reservoir structure, and the echo state property that this work inherits.","marker":"[22]"},{"why":"Introduces LASSO regression, which is used here to select the significant delay variables.","marker":"[25]"},{"why":"Supplies the projection algorithm used for the online update of the output weights.","marker":"[28]"},{"why":"Provides the debutanizer column benchmark and the auxiliary-variable formulation used in the soft-sensing experiment.","marker":"[29]"}],"fun_headline_variants":["Hybrid regularized RSCN: universal approximation, top test scores","LASSO-L2 RSCN beats all models on every test set","LASSO selects, L2 fits: recurrent SCN wins nonlinear tasks"],"cache_read_input_tokens":19328,"weakest_assumption_plain":"The proof of Theorem 1 assumes that the global ridge-regression solution always has unregularized squared error no larger than the constructive one-node update; this can fail when the current network already fits the training data well, because the L2 penalty shrinks every weight and can increase the unregularized error.","fun_headline_variants_meta":{"raw":{"variants":["Hybrid regularized RSCN: universal approximation, top test scores","LASSO-L2 RSCN beats all models on every test set","LASSO selects, L2 fits: recurrent SCN wins nonlinear tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000786,"raw_usage":{"total_tokens":3445,"prompt_tokens":901,"completion_tokens":2544,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":517,"completion_tokens_details":{"reasoning_tokens":2482}},"tokens_in":517,"tokens_out":2544,"duration_ms":19104,"temperature":1.0,"reasoning_tokens":2482,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:35:21.748389+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a case with two training samples and one existing reservoir node whose current output weight already makes the residual exactly zero, then add one candidate node and compute both $\\|\\tilde{e}_{N+1}\\|^2$ from the global ridge solution (17)-(18) and $\\|e_{N+1}\\|^2$ from the single-node update (15); the proof needs $\\|\\tilde{e}_{N+1}\\|^2 \\le \\|e_{N+1}\\|^2$, and a simple numerical example will show whether that inequality holds for any $C>0$.","supporting_citations":[{"cited_title":"Stochastic configuration networks: Fundamentals and algorithms,","cited_arxiv_id":null,"evidence_quote":"Introduces the stochastic configuration network and the supervisory inequality for random parameter assignment that the RSCN builds on."},{"cited_title":"Regression shrinkage and selection via the lasso,","cited_arxiv_id":null,"evidence_quote":"Introduces LASSO regression, which is used here to select the significant delay variables."},{"cited_title":"Adaptive filtering prediction and control,","cited_arxiv_id":null,"evidence_quote":"Supplies the projection algorithm used for the online update of the output weights."},{"cited_title":"Soft sensors for product quality monitoring in debutanizer distillation columns,","cited_arxiv_id":null,"evidence_quote":"Provides the debutanizer column benchmark and the auxiliary-variable formulation used in the soft-sensing experiment."}],"review_version":1}