{"id":"175ba800-6c0c-4320-ae23-b09f06a39969","arxiv_id":"1909.01961","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"An iterative threshold-based node selection scheme for data-driven randomized feedforward networks produces more compact architectures and faster error reduction than the non-constructive baseline.","lead":"This paper proposes a constructive variant of a data-driven randomized neural network algorithm: it adds hidden neurons one at a time and keeps only those that reduce training error beyond an adaptive threshold. The result is a smaller network that reaches the same accuracy as the non-constructive version in tests on four regression problems.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No test shows that nodes built from the original target can represent the residual; the constructive claim rests on an untested assumption.","rationale":"The reader's verdict already identifies this untested assumption, and my reading of Algorithm 1 confirms it is the point where the central claim is least secure. The four empirical examples show the assumption can hold, but they do not establish it, and no residual-based comparison is reported. The concern about unsupported state-of-the-art baselines is also real, but it concerns the paper's secondary transitive claim rather than the core D-DM versus CD-DM comparison. I therefore keep the conditional verdict.","tokens_in":8450,"tokens_out":15005,"duration_ms":176680,"concrete_test":"Implement a residual-based variant (CD-DM-R) that keeps Algorithm 1 unchanged except that step (a) samples x* with probability proportional to the current absolute residual and step (c) fits the hyperplane to the residual y_l - Y'_l instead of to y_l. Tune m, k', θ, and Q on the same four datasets over 100 trials. If CD-DM-R needs fewer nodes or achieves lower test RMSE, the original candidate-generation assumption is limiting; if the two variants perform similarly, the assumption is empirically adequate.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The acceptance threshold in Algorithm 1 is the only residual-dependent step. Candidate nodes are always generated from the original target: step (a) draws x* uniformly, step (c) fits a hyperplane to the original outputs y in the neighborhood Ψ(x*), and step (e) sets the bias from x*. The hidden layer is therefore a filtered subset of a fixed family of target-derived sigmoids, not a residual-adaptive basis. The paper's compactness claim assumes that the residual after earlier accepted nodes can be approximated by further nodes of this same fixed family. That assumption is not self-evident and is not tested: no experiment varies candidate generation to use the current residual. If the residual has structure not present in the original local slopes, the method can only lower θ toward zero and accept weakly useful nodes, and the 'significant neuron' interpretation breaks down. This is a load-bearing gap because the constructive mechanism is the paper's claimed contribution.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CD-DM, a constructive extension of the data-driven random hidden node method D-DM. The algorithm iteratively generates candidate sigmoid hidden nodes by sampling a training point x*, fitting a local hyperplane to the original target values in its neighborhood, and setting the node's weight vector and bias from that hyperplane. A candidate is accepted only if it decreases the training RMSE by more than an adaptive threshold; the threshold is halved whenever no node is accepted for Q consecutive attempts. Experiments compare CD-DM with D-DM on one univariate synthetic function, one bivariate synthetic function, and three real-world datasets, reporting medians and interquartile ranges over 100 runs. The authors claim faster convergence, more compact architectures, and, via a transitive comparison with an unpublished companion paper, superiority over standard and state-of-the-art randomized learning methods.","tokens_in":8608,"tokens_out":6434,"duration_ms":69228,"significance":"If the empirical claims hold, the paper makes a useful practical contribution to randomized feedforward learning: a data-dependent basis combined with greedy error-based selection can produce smaller networks than adding an entire random projection layer at once. The experimental protocol has notable strengths: 100 independent trials, separate test sets, median and IQR reporting, and cross-validation for the main hyperparameters m and k'. However, the central constructive mechanism is only partially validated, the state-of-the-art comparison rests on an unpublished anonymous baseline, and the accuracy differences are not significance-tested. The contribution is incremental rather than foundational, but it is of interest to researchers working on randomized neural networks and deserves further scrutiny.","major_comments":[{"comment":"Candidate hidden nodes are generated from the original target values y, not from the current residual: step (c) fits the hyperplane to the neighborhood of x* using the original outputs, and step (k) is the only residual-dependent operation. The central compactness claim therefore assumes that the residual after previously accepted nodes can be approximated by further nodes drawn from the same target-derived family. This assumption is not argued for and is not tested; for example, no experiment generates candidates from the current residual y - Y'. If the residual has structure absent from the original local slopes, the procedure can only halve theta until weakly useful nodes pass the threshold, and the interpretation that accepted neurons are 'significant' would break down. I ask the authors to either supply a supporting argument or add an experimental variant with residual-based candidate generation; at minimum, this limitation should be stated explicitly.","section":"Algorithm 1, steps (a)-(e)"},{"comment":"The conclusion that CD-DM outperforms standard and state-of-the-art methods is a transitive argument based entirely on (Anonymous 2019a), an unpublished manuscript that the reader cannot access. The core node-generation equations (3)-(4) are also taken from that same source. To support the headline claim, the authors should either run published baselines directly on the four benchmark problems or restrict the claim to 'CD-DM is more compact and faster-converging than D-DM', with published references used only for qualitative context.","section":"Simulation study, final paragraph"},{"comment":"The RMSE differences between CD-DM and D-DM are reported as medians with interquartile ranges, but no paired significance tests are provided. For Stock the intervals are 0.0285 +/- 0.0028 versus 0.0265 +/- 0.0014, and for Concrete 0.0770 +/- 0.0055 versus 0.0748 +/- 0.0034; these intervals overlap considerably. The compactness and convergence-speed claims are credible, but the accuracy advantage should be established by paired per-run tests, or the paper should state that no consistent accuracy improvement is claimed.","section":"Simulation study, Table 1 and Fig. 4"}],"minor_comments":[{"comment":"The initialization RMSE0 = 1 assumes the target outputs have been normalized to [0,1]; this normalization should be stated explicitly before the algorithm is presented.","section":"Algorithm 1"},{"comment":"The text says the threshold theta is 'reduced by half', but because theta is initialized to a negative value, the operation theta = theta/2 increases it toward zero. The sign convention should be stated explicitly to avoid confusion.","section":"Abstract and Constructive Approach section"},{"comment":"The phrase 'faster convergence' is always measured in terms of the number of hidden nodes; no wall-clock training times are reported. The authors should state that the advantage is in hidden-node count and not claim a runtime advantage without timing experiments.","section":"Simulation study"},{"comment":"The discussion of constructive randomized networks is thin given that construction is central to the paper; citing and comparing with incremental extreme learning machines and stochastic configuration networks would help position the contribution.","section":"Related work"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's heavy reliance on the unpublished anonymous companion paper (Anonymous 2019a) makes independent verification difficult. I would advise the editor to ensure that this companion paper is made available to reviewers, or to require a revision that removes the dependence by running direct baseline comparisons. The self-citation pattern is not itself problematic, but it currently blocks assessment of the novelty and of the state-of-the-art claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Know this paper mostly for what it is: a small, clearly specified engineering extension of Dudek's own D-DM randomized single-hidden-layer scheme. The new piece is a constructive loop with an error-reduction acceptance threshold that halves (toward zero) when no node is accepted for Q trials. On the four regression problems tested, CD-DM reaches the same or better test error with 2–5x fewer hidden nodes. That result is internally credible: 100 runs, separate test sets, CV for m and k', median+IQR. If you work on randomized neural nets, the pseudocode is enough to reimplement.\n\nWhat is genuinely new: the threshold adaptation schedule is concrete, and the comparison against D-DM is clean. The paper also reports percentile bands, which is more than many papers in this area.\n\nNow the soft spots, in rough order of importance. First, the core node-generation equations (a=4a', b=-a^T x*) and the state-of-the-art comparison are both taken from the same unpublished anonymous predecessor. The paper cannot be evaluated on its central accuracy claim until that paper appears or direct baselines are run. That is not a minor issue. Second, the stress-test concern is real: candidates are always fitted to the original target's local slopes, never to the current residual. The acceptance filter is the only residual-dependent step. On the tested functions this works, but the 'significant neuron' story would be stronger with an experiment that generates candidates from the residual, or at least a discussion of why original-target candidates suffice. Third, no direct head-to-head with SCN or block-increment SCN, despite citing them; the text claims superiority only transitively. Fourth, minor: no code, no wall-clock times, and k' is reused from D-DM without revalidation.\n\nOverall: the central internal comparison holds up; the broad external claim does not. The paper is a candidate for a serious referee, but acceptance should hinge on the author addressing the self-citation gap and adding at least one direct comparison with a non-self SOTA method. I wouldn't cite it until the D-DM paper is public, but if you're in the randomized-FNN niche, it's worth a reading-group slot.","headline":"A clean, incremental constructive extension of the author's own D-DM that shows real compactness gains on four regressions, but its headline SOTA claim rests on an unpublished self-cited predecessor and an untested residual assumption.","tokens_in":9154,"tokens_out":2272,"would_cite":false,"duration_ms":23139,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Single-hidden-layer networks can be grown by keeping only sigmoids that reduce training error.","keywords":["randomized learning","feedforward neural networks","random hidden nodes","data-driven parameter generation","constructive network architecture","sigmoid activation","regression"],"falsifier":"Run the paper's 100-trial protocol on function (6) with the acceptance filter disabled, so every candidate is kept while all other settings are unchanged; if CD-DM does not need far fewer nodes than D-DM to reach RMSE ≈ 0.00068 on the held-out test points, the filter is not the source of the reported compactness.","tokens_in":8173,"feed_emoji":"🧠","tokens_out":12541,"duration_ms":120050,"temperature":0.7,"pith_summary":"Randomized feedforward networks avoid gradient training by fixing random hidden-node parameters and solving only for output weights, but accuracy depends on the interval those random parameters are drawn from. This paper proposes building the hidden layer one candidate neuron at a time instead of drawing a full random layer at once: each S-shaped sigmoid is placed at a random training point, its slope is set from the local slope of the target function, and it is kept only if adding it lowers training error by more than an adaptive threshold. The threshold starts strict, admits coarse-shape neurons, and is halved after every stretch of rejected candidates so detail neurons enter later. The paper reports that this constructive version converges faster, uses two to five times fewer hidden nodes on real datasets, and on a one-dimensional benchmark reaches with 33 nodes the accuracy the original method reaches with 250 nodes. If the claim is right, randomized networks become smaller, faster to train, and less sensitive to an arbitrarily chosen random-parameter interval.","feed_headline":"Error-gated node selection shrinks random networks 2-5x","feed_subtitle":"Keeping only hidden nodes that cut training error reaches the same accuracy with far fewer neurons.","key_machinery":"The central mechanism is an adaptive acceptance threshold $\\theta$ wrapped around a data-driven candidate generator. Each candidate sigmoid is anchored at a random training point $x^*$; a hyperplane fitted to the point and its $k$ nearest neighbours gives local slope coefficients $a'_j$, and the sigmoid weights are set to $a_j = 4a'_j$ with bias $b = -a^T x^*$, placing the sigmoid's inflection point at $x^*$. The candidate is added only if the training error decreases by at least $\\theta$; after $Q$ consecutive rejections $\\theta$ is halved, so early stages admit only coarse-shape nodes and later stages admit detail nodes. The output weights are re-estimated by a standard linear least-squares fit through the pseudoinverse of the hidden-layer output matrix after every accepted node.","core_discovery":"The central claim is that an acceptance filter on incremental error reduction turns a randomized single-hidden-layer network into a compact constructive model without sacrificing accuracy. In the proposed CD-DM, candidate sigmoids are generated exactly as in the original data-driven method, but instead of being added wholesale they are admitted only when the training root mean square error decreases by at least $\\theta$; $\\theta$ is halved once $Q$ consecutive candidates fail, so the network first locks in nodes that capture the target's rough shape and then fills in detail. The reported evidence is that on function (6) a 33-node CD-DM reaches about the same test error ($0.00068$) as a 250-node D-DM, and on Stock, Concrete and Compactiv the constructive version needs two to five times fewer nodes and has lower median test error with less dispersion across runs. The paper presents this as a general statement: architecture size should be governed by target-function complexity, not by a fixed draw of random parameters.","pith_inferences":["The paper does not test the natural variant in which candidate hyperplanes are fitted to the current residual rather than to the original target; if that variant worked, the acceptance threshold would be a general-purpose residual-refinement rule rather than a filter specific to one generator.","The threshold-halving schedule is a coarse-to-fine schedule: the construction suggests early accepted nodes encode the target's broad shape and later nodes encode detail, although the paper does not measure this decomposition directly.","The same error-reduction acceptance rule should transplant to classification by substituting a classification loss, a direction the paper lists as future work."],"forward_implications":["The random-parameter interval ceases to be a tuning problem: weights and biases come from local data geometry, and model selection reduces to choosing the final node count and the neighborhood size.","Because only accepted nodes enter the hidden-layer matrix, the least-squares solves stay small, so training cost and prediction cost are lower than for a one-shot random layer of the same accuracy.","The lower run-to-run dispersion reported for CD-DM implies the final model depends less on which random training points happen to be drawn.","On the three real datasets, using two to five times fewer nodes makes cross-validated model selection cheaper and the deployed network smaller."],"supporting_citations":[{"why":"Supplies the base data-driven node generator, from which CD-DM is built and against which all reported comparisons are made.","marker":"Anonymous 2019a"},{"why":"Provides an alternative random-parameter generation strategy that the paper's comparison context treats as a recent alternative.","marker":"Anonymous 2019b"},{"why":"Defines a constructive randomized-network algorithm that searches for parameter intervals; CD-DM is positioned as an interval-free alternative.","marker":"Wang and Li 2017"},{"why":"Offers another constructive randomized-network algorithm with block increments, used as a recent point of comparison for constructive approaches.","marker":"Dai et al. 2019"}],"fun_headline_variants":["Adaptive threshold trims random neural networks","Node acceptance filter builds compact nets","Half-threshold gating yields leaner random nets","Constructive random nets: 2-5x fewer nodes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that repeatedly sampling hyperplanes fitted to the original target function's local neighborhoods, and keeping only candidates that lower training error, is enough to approximate the residual detail that earlier nodes missed; the paper does not test a residual-based generator that would make this assumption unnecessary.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive threshold trims random neural networks","Node acceptance filter builds compact nets","Half-threshold gating yields leaner random nets","Constructive random nets: 2-5x fewer nodes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000183,"raw_usage":{"total_tokens":1326,"prompt_tokens":970,"completion_tokens":356,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":586,"completion_tokens_details":{"reasoning_tokens":297}},"tokens_in":586,"tokens_out":356,"duration_ms":4273,"temperature":1.0,"reasoning_tokens":297,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:03:45.815912+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the paper's 100-trial protocol on function (6) with the acceptance filter disabled, so every candidate is kept while all other settings are unchanged; if CD-DM does not need far fewer nodes than D-DM to reach RMSE ≈ 0.00068 on the held-out test points, the filter is not the source of the reported compactness.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines a constructive randomized-network algorithm that searches for parameter intervals; CD-DM is positioned as an interval-free alternative."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Offers another constructive randomized-network algorithm with block increments, used as a recent point of comparison for constructive approaches."}],"review_version":1}