{"id":"12ac7bbf-048f-46a6-aca0-a47a1cefb44e","arxiv_id":"2411.08544","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"RMPI-SCN replaces SCN-III's lower-bound candidate selection with an exact recursive Moore-Penrose criterion and reports better training RMSE on nine of ten datasets.","lead":"An exact error-reduction test lets a new variant of Stochastic Configuration Networks pick better random basis functions during incremental training, improving learning on most benchmarks tested. The paper is a focused algorithmic contribution to a class of randomized neural networks used for fast large-scale regression.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central causal claim is not isolated: RMPI-SCN changes the selection rule, the r-schedule, and the MP-update path simultaneously, so the observed gains cannot be attributed to the new supervisory mechanism without an ablation.","rationale":"The reader's CONDITIONAL verdict is well calibrated. Theorem 2's algebra is sound: p_L = h - H_{L-1}H_{L-1}^dagger h is the orthogonal component of the candidate, Eq. (30) gives the exact post-fit residual, and selecting its minimum is genuinely one-step optimal. The paper deserves credit for this clean derivation and for a consistent empirical pattern against SCN-III. My stress-test concern is causal attribution rather than correctness of the theorem. RMPI-SCN changes several algorithmic ingredients at once; without an ablation, the paper may overclaim that the exact selection mechanism causes the improvement. The proposed ablation is inexpensive and would settle the issue. Separately, the alpha-schedule statement around Eq. (45) is also misstated: lim_{L->infty} r^{(1+1/L)^alpha} = r, not r^alpha, although this is secondary because convergence with rate r<1 would still hold. Given that these issues are addressable with additional experiments and corrections, I would keep the conditional posture rather than reject.","tokens_in":20361,"tokens_out":19944,"duration_ms":201657,"concrete_test":"On DB1, DB4, and DB8, run three variants inside the same RMPI skeleton with identical r_L schedule, identical lambda-expansion sequence, identical recursive H-dagger update, and identical random candidate batches: (a) exact-residual argmin; (b) original xi argmax; (c) uniform random choice among candidates satisfying Eq. (50). Compare training/test RMSE and training-residual curves over 100 runs. If (b) or (c) matches (a), the exact selector is not the cause of the gains; if (a) is strictly better, the causal claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"RMPI-SCN differs from SCN-III in at least three coupled ways: it selects candidates by the exact one-step residual (Eqs. 52, 55-57) instead of the xi lower bound; it replaces the inner r-search with the schedule r_L = r^((1+1/L)^alpha) and a different lambda-expansion policy; and it maintains H-dagger by recursive Greville updates. The paper attributes the empirical gains to the exact residual evaluation, but no experiment isolates this rule from the other two changes. The theoretical result only shows that, among the current candidate batch, the exact selector minimizes the immediate training residual; it does not logically imply better final training or test RMSE relative to SCN-III's heuristic. Because the comparison confounds the selector with the r_L scheduler and the recursive-update path, the observed improvements cannot be cleanly attributed to the new supervisory mechanism. This gap is load-bearing: if a xi-based or random-but-valid selector inside the same RMPI scheduler produces the same convergence and RMSE, the paper's main contribution collapses to an implementation detail. The absence of significance tests further weakens the small differences on datasets such as DB3, DB5, DB7, and DB9.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper analyzes the supervisory mechanism of SCN-III, arguing that the standard lower-bound criterion ξ in Eq. (10) can select suboptimal random basis functions. It derives necessary and sufficient conditions (Theorem 2, Corollaries 2 and 3) for the one-step training residual to satisfy ||Y - H_L H_L^† Y|| ≤ √r ||e_{L-1}||, using Greville's recursive pseudoinverse and the projection residual p_L = h - H_{L-1} H_{L-1}^† h. The paper then proposes RMPI-SCN, which selects candidate nodes by the exact one-step residual, replaces SCN-III's inner r-search with a scheduled rate r_L = r^{(1+1/L)α}, and maintains H_L^† recursively. The empirical section compares RMPI-SCN with SCN-III, RVFL, and MLP-BP on ten regression datasets.","tokens_in":20518,"tokens_out":10779,"duration_ms":72941,"significance":"The theoretical core is sound and useful: Theorem 2 reduces candidate evaluation to vector operations involving p_L and gives a clean orthogonal-projection characterization of the one-step residual reduction; the proof does not depend on the data and introduces no tunable constants other than r. If validated in isolation, the new selection rule would be a meaningful improvement over the lower-bound-based selection in SCN-III. The main weakness is that the empirical claims are not isolated: RMPI-SCN changes the selector, the r-schedule, and the update path simultaneously, and no ablation, statistical significance testing, or comparison with the closest improved-SCN baselines is provided.","major_comments":[{"comment":"RMPI-SCN differs from SCN-III in at least three coupled design changes: the candidate score (the exact one-step residual ξ_j instead of SCN-III's lower-bound ξ), the scheduled learning rate r_L that eliminates the inner r-search, and the recursive Greville update of H^†. The paper attributes the better results in Table III to the new 'accurate' supervisory mechanism, but no experiment isolates this rule from the other two modifications. Add at least two ablations on the same candidate draws: exact-residual selection with SCN-III's r-search, and SCN-III's ξ selection with the r_L schedule. Without these, the observed gains cannot be attributed to the new evaluation mechanism.","section":"Section III.C, Eqs. (47), (55)-(57), (58)-(60)"},{"comment":"The paper states that RMPI-SCN's faster convergence is 'not an experiment coincidence but a mathematical consequence' because it selects the candidate minimizing the current residual. This overreaches: minimizing the immediate one-step residual among the current candidates gives no logical guarantee about the final training or test RMSE, since the greedy choice affects future candidate distributions and the residual path. The reported exceptions in Table III (e.g., DB5 training RMSE, where RMPI-SCN is slightly worse than SCN-III, and DB10 test RMSE, where RVFL is best) are acknowledged but not explained. Either provide a theoretical argument for the greedy trajectory or replace the causal claim by the directly supported statement that RMPI-SCN minimizes the one-step residual.","section":"Section IV, text around Figs. 4 and 8"},{"comment":"The 100-run comparisons are reported only as means and standard deviations; no statistical significance tests are given. For several datasets the differences are small relative to the reported variation (e.g., DB3 test RMSE: 3.9136 ± 0.0243 vs. 3.9635 ± 0.0171; DB5; DB7; DB9), so the claim that RMPI-SCN 'outperforms' SCN-III is not established. Report paired tests or bootstrap confidence intervals over the 100 repetitions, preferably using identical candidate draws for both methods.","section":"Table III and Section IV"},{"comment":"The paper claims reduced computational complexity and enhanced scalability, but the reported wall-clock times are not consistent with that claim: RMPI-SCN is slower than SCN-III on DB4 (43.031 s vs. 24.373 s), DB6 (0.160 s vs. 0.130 s), DB9 (0.381 s vs. 0.221 s), and DB10 (0.334 s vs. 0.198 s). Since RMPI-SCN also uses fewer nodes on some datasets, a fair complexity comparison would require operation counts or runtime to reach a fixed residual threshold. Without such an analysis, the efficiency claim is not supported by the experiments.","section":"Section IV, Table III and abstract"},{"comment":"The experimental evaluation does not include the closest prior-art improvements to SCN's supervisory mechanism, in particular Refs. [16], [17], [18], and [25]. These methods directly address the same lower-bound limitation that motivates RMPI-SCN. The paper should compare against these methods on the same benchmarks, or explicitly justify their exclusion; without such comparisons, the contribution relative to the state of the art is not established.","section":"Section IV, comparisons"}],"minor_comments":[{"comment":"The pseudocode sets r* = r^{(1+1/L)γ}, but the hyperparameter introduced in Section III.B and used in Corollary 3 is α; please align the notation.","section":"Algorithm 1, line 10"},{"comment":"The abstract says the method works 'without requiring the computation of the Moore-Penrose inverse of the output matrix', but Eqs. (58)-(60) recursively maintain H_L^†. Please rephrase to 'without recomputing H_L^† from scratch for each candidate'.","section":"Abstract and Section III.A"},{"comment":"The sentence 'The datasets DB3-DB10 are benchmark datasets: CCPP, Superconduct, Delta Ail, Stock, Concrete, and CCPP' repeats CCPP and does not match Table II; the list should be corrected to match DB3-DB10.","section":"Section IV, dataset list"},{"comment":"The pseudocode line 'eL = HLH†LT − T' contains a typo; it should read 'eL = HLH†LY − Y'.","section":"Algorithm 1, line 17"},{"comment":"The definition of ξ_j in Eqs. (56)-(57) is not explicitly derived from the minimization of ||Y - H_L H_L^† Y|| in Eq. (52); a short derivation or explanatory sentence would aid reproducibility.","section":"Section III.C, Eqs. (56)-(57)"},{"comment":"Condition Eq. (29) is redundant: because p_L is orthogonal to range(H_{L-1}) and Y - e_{L-1} belongs to range(H_{L-1}), we have ⟨Y - e_{L-1}, p_L⟩ = 0, so Eq. (29) reduces to the nonnegativity already implied by Eq. (28). This does not affect the correctness of the theorem, but the statement can be simplified.","section":"Theorem 2, Eq. (29)"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is Theorem 2: a necessary and sufficient condition for ||Y - H_L H_L^† Y|| ≤ sqrt(r)||e_{L-1}||, expressed via the orthogonal component p_L = h - H_{L-1}H_{L-1}^†h. The derivation using Greville's recursion checks out as far as I followed it, and it yields a practical way to evaluate candidates without forming the new pseudoinverse each time. That is a real contribution, and it fixes a genuine weakness: the ξ-based criterion in SCN-III is only a lower bound for the one-step residual reduction, so maximizing ξ does not necessarily pick the best candidate. The new condition is exact for that one-step goal, which is a clean theoretical improvement.\n\nThe soft spots are all in the empirical section. Most importantly, RMPI-SCN changes three things at once: the selection rule, the r_L schedule, and the recursive update path. No ablation isolates the new selector, so the observed gains in training and test RMSE cannot be cleanly attributed to it. The theoretical result only says the exact selector minimizes the immediate residual among the candidate batch; it does not promise better final generalization. Second, the paper does not compare against the most relevant improved SCN baselines (refs [16] and [18]), which also revise the supervisory mechanism. Third, the abstract and introduction claim reduced computational complexity, but the tables show RMPI-SCN is slower than SCN-III on most datasets; the paper acknowledges this in the text, but it undercuts the scalability claim. There is also a notational inconsistency in the definition of e_{L-1} between Theorem 2 and the algorithm, and the sign in Eq. (57) looks wrong under one reading (it may be consistent with the flipped residual definition, but as written it will confuse). No significance tests are reported, and some differences on DB3, DB5, DB7, and DB9 are small enough that 100 runs without a test statistic do not settle them.\n\nNone of this kills the core contribution. The math is solid, the idea is clearly motivated, and the weaknesses are addressable: add an ablation, compare with the improved SCN variants, report significance, fix the notation, and release code or full hyperparameters. I would want those before trusting the empirical claims, but the theoretical result stands on its own.\n\nThis paper deserves a serious referee. It is exactly the kind of work that should go through peer review: a correct and checkable theoretical improvement with an evaluation that needs strengthening. I would take it with the expectation of major revisions.","headline":"Correct and useful exact one-step selection condition for SCN-III, but the empirical evaluation confounds the new selector with other changes and misses the closest prior baselines.","tokens_in":21103,"tokens_out":3825,"would_cite":true,"duration_ms":36497,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proves an exact residual test for candidate basis functions in SCN-III, replacing a lower-bound score, and shows that the resulting RMPI-SCN converges faster and learns better on ten regression benchmarks.","keywords":["stochastic configuration networks","randomized learning","incremental learning","Moore-Penrose inverse","supervisory mechanism","basis function selection","regression"],"falsifier":"On a regression dataset with strongly overlapping basis functions, run RMPI-SCN and SCN-III to the same node budget and compare held-out RMSE across many seeds; if SCN-III is not worse on a majority of datasets, the claimed advantage of exact one-step selection is falsified.","tokens_in":20085,"feed_emoji":"📉","tokens_out":9858,"duration_ms":90500,"temperature":0.7,"pith_summary":"Stochastic Configuration Networks are randomized neural networks that grow one hidden node at a time, accepting only candidate basis functions that meet a residual-shrinking test. This paper identifies a mismatch in the standard SCN-III test: its score $\\xi$ measures how much the candidate would reduce the error if only its own output weight were updated, but SCN-III actually refits all output weights, so the score is only a lower bound on the candidate's true value. The paper derives a necessary-and-sufficient condition for a candidate to achieve $\\|e_L\\| \\le \\sqrt{r}\\|e_{L-1}\\|$ in SCN-III, based on a recursive Moore-Penrose inverse calculation, and uses it to build RMPI-SCN, a training scheme that selects the candidate with the smallest exact one-step residual. Simulations on ten regression datasets show RMPI-SCN reaches lower training and test error with fewer hidden nodes than SCN-III on most benchmarks.","feed_headline":"Exact residual test speeds up stochastic configuration networks","feed_subtitle":"A recursive inverse lets each new hidden node be chosen by true error reduction, improving ten regression benchmarks.","key_machinery":"The load-bearing identity is the recursive Moore-Penrose update. For an existing hidden-layer matrix $H_{L-1}$ and a candidate output vector $h$, Greville's formulas give $H_L^\\dagger$ in terms of $H_{L-1}^\\dagger$, $d_L = H_{L-1}^\\dagger h$, and $b_L$; substituting into the projection $Y - H_L H_L^\\dagger Y$ collapses the new residual to $e_{L-1} - p_L\\tau_L$, with $p_L = h - H_{L-1}H_{L-1}^\\dagger h$ and $\\tau_L = \\langle Y,p_L\\rangle/\\|p_L\\|^2$. This turns residual evaluation into a small inner-product computation and yields the necessary-and-sufficient inequalities that define the new supervisory mechanism. The same recursive update is reused inside RMPI-SCN to maintain $H_L^\\dagger$ as nodes are added, so no full pseudoinverse of the growing output matrix is recomputed.","core_discovery":"The central claim is that the supervisory mechanism of SCN-III can be made exact. Theorem 2 states that for a candidate basis function $h$, the inequality $\\|Y - [H_{L-1},h][H_{L-1},h]^\\dagger Y\\| \\le \\sqrt{r}\\|e_{L-1}\\|$ holds if and only if $p_L = h - H_{L-1}H_{L-1}^\\dagger h$ is nonzero, $\\langle e_{L-1},p_L\\rangle^2/\\|p_L\\|^2 \\ge (1-r)\\|e_{L-1}\\|^2$, and $\\|\\langle Y - e_{L-1},p_L\\rangle\\| \\le \\sqrt{\\delta_L}$, where $\\delta_L = \\langle e_{L-1},p_L\\rangle^2 - (1-r)\\|p_L\\|^2\\|e_{L-1}\\|^2$. Because $p_L$ and the scalar $\\tau_L = \\langle Y,p_L\\rangle/\\|p_L\\|^2$ can be computed from the already-stored $H_{L-1}^\\dagger$, the true one-step residual is available without forming the Moore-Penrose inverse of the augmented matrix. The authors fold this into RMPI-SCN, which also uses a size-dependent learning rate $r_L = r^{(1+1/L)^\\alpha}$, and report that it consistently selects the most effective random basis function at each iteration, converges faster, and improves training and test error on most of ten benchmarks.","pith_inferences":["The greedy one-step residual is not obviously the best proxy for final generalization; the paper's own tables show RMPI-SCN trailing SCN-III on one training metric (DB5) and one test metric (DB10), so the advantage may be dataset-dependent.","The recursive pseudoinverse update is exact in exact arithmetic, but no numerical-stability analysis is given for hundreds of accumulated updates; a QR- or SVD-based refresh might be needed in very large networks.","The same residual identity could be carried into recurrent, 2D, and deep SCN variants, since Theorem 2 only treats single-hidden-layer feedforward networks; the authors themselves list these as future work."],"forward_implications":["SCN-III implementations can replace the lower-bound score $\\xi$ with the exact one-step residual, removing the need to search over $r$ in an inner loop and the need to compute $[H_{L-1},h]^\\dagger$ for every candidate.","Because smaller $r$ values remain usable for longer, the algorithm can keep the uniform distribution's support tighter when appropriate and expand it more accurately when the exact criterion fails.","The adaptive schedule $r_L = r^{(1+1/L)^\\alpha}$ preserves the geometric convergence guarantee, with $\\limsup \\|e_L\\|/\\|e_{L-1}\\| \\le r^\\alpha$, while avoiding the practice of forcing $r$ close to one.","If the central claim is correct, switching to the exact residual test should reproduce the reported gains: lower training RMSE on nine of ten datasets, lower test RMSE on most, and fewer nodes (e.g., 268 vs 398 on DB4)."],"supporting_citations":[{"why":"Defines the SCN framework, SCN-I and SCN-III, and the lower-bound residual inequality that the paper's new conditions replace.","marker":"[14]"},{"why":"Supplies the recursive Moore-Penrose inverse formulas used to express the residual after adding a candidate basis function.","marker":"[27]"},{"why":"Establishes the random-basis-function approximation view that SCNs build on and provides the RVFL baseline for comparison.","marker":"[5]"},{"why":"An earlier study of SCN inequality constraints that motivates tightening the supervisory mechanism.","marker":"[16]"},{"why":"A recent improved supervisory mechanism for SCNs, the immediate line of work this paper extends.","marker":"[18]"}],"fun_headline_variants":["Exact residual test picks better hidden nodes","No Moore-Penrose inverse needed for SCNs","True error reduction guides SCN basis choice","RMPI-SCN: exact scoring speeds learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's results rest on the assumption that always adding the hidden node that reduces the current training error the most will also produce a better final model, an assumption that is supported empirically on most datasets but not proven.","fun_headline_variants_meta":{"raw":{"variants":["Exact residual test picks better hidden nodes","No Moore-Penrose inverse needed for SCNs","True error reduction guides SCN basis choice","RMPI-SCN: exact scoring speeds learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000168,"raw_usage":{"total_tokens":1354,"prompt_tokens":1134,"completion_tokens":220,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":750,"completion_tokens_details":{"reasoning_tokens":160}},"tokens_in":750,"tokens_out":220,"duration_ms":2837,"temperature":1.0,"reasoning_tokens":160,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T21:32:45.118509+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a regression dataset with strongly overlapping basis functions, run RMPI-SCN and SCN-III to the same node budget and compare held-out RMSE across many seeds; if SCN-III is not worse on a majority of datasets, the claimed advantage of exact one-step selection is falsified.","supporting_citations":[{"cited_title":"Wang and M","cited_arxiv_id":null,"evidence_quote":"Defines the SCN framework, SCN-I and SCN-III, and the lower-bound residual inequality that the paper's new conditions replace."},{"cited_title":"Greville, ”Some applications of the pseudoinverse of a matrix,” SIAM Review, vol","cited_arxiv_id":null,"evidence_quote":"Supplies the recursive Moore-Penrose inverse formulas used to express the residual after adding a candidate basis function."},{"cited_title":"Igelnik and Y .-H","cited_arxiv_id":null,"evidence_quote":"Establishes the random-basis-function approximation view that SCNs build on and provides the RVFL baseline for comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"An earlier study of SCN inequality constraints that motivates tightening the supervisory mechanism."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"A recent improved supervisory mechanism for SCNs, the immediate line of work this paper extends."}],"review_version":1}