{"id":"8975a215-8333-4eaf-9acd-2ed845019c26","arxiv_id":"1908.05864","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A modified generation scheme for random weights and biases places sigmoid inflection points inside the input hypercube and optionally makes slope angles uniform, improving fit on two synthetic benchmarks.","lead":"This paper proposes a better way to choose the random weights and biases in single-layer neural networks with random hidden nodes. The new method places each sigmoid's steepest part inside the input region and can make the slope angles uniform, which the author says reduces the number of nodes needed and overfitting.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central improvement assumes saturated sigmoids are useless; if false, forcing inflection hyperplanes into H via Eq. (6) need not help and may reduce basis diversity.","rationale":"The bias construction itself is algebraically sound: Eq. (6) indeed puts x* on the inflection hyperplane, and Eq. (9) correctly expresses the coordinate-wise slope angle. The weak point is the approximation-theoretic premise used to argue this should improve accuracy. A linear combination of sigmoids can use saturated units to build constants and smooth trends; the paper's own low-node-count examples do not isolate this effect. The proposed check would settle whether the benefit survives outside the high-frequency regime. The reader's weakest_assumption already identifies this heuristic, and I agree. I also note the hyperparameter selection issue in Sec. 4, but it is secondary: even with clean selection, the general claim would remain unproven without the frequency-controlled experiment. Since the construction is valid and the evidence is suggestive but not decisive, the CONDITIONAL verdict stands.","tokens_in":5870,"tokens_out":12122,"duration_ms":128407,"concrete_test":"Run a controlled comparison on synthetic targets with controlled frequency content, e.g., g_k(x)=sin(2πkx) for k=0,1,2,4,8 plus a linear trend, using the same weight distribution and output least-squares solver for (A) standard biases b∼U(-u,u) and (B) proposed biases b=-a x*, x*∼U(0,1). For each node count, select u or alpha_min on a separate validation fold, then evaluate test RMSE on a held-out set over 100 trials. If (B) does not beat (A) for low-k or smooth targets, the 'saturated fragments are useless' premise is false and the method's benefit is task-specific rather than general.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2 justifies the bias rule by asserting that 'saturated fragments of these sigmoids ... are useless for modeling nonlinear TFs' and that inflection-centered fragments are the useful ones. This premise is load-bearing: if saturated or gently varying hidden units can contribute to a linear combination, e.g., as near-constant offsets or low-frequency components, then placing every inflection hyperplane inside H by Eq. (6) need not improve accuracy and could reduce the diversity of basis functions available to the output layer. The paper's experiments use only two synthetic targets, Eqs. (1) and (10), both dominated by high-frequency fluctuations over [0,1], so they cannot detect whether the proposed bias placement helps on smoother targets. A compounding issue is that Sec. 4 chooses the optimal u or alpha_min per node count from a grid without stating whether the selection uses a validation split; the reported PMalpha curves are therefore envelopes over hyperparameters and may overstate the improvement. Together these leave the central performance claim conditional.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies how random weights and biases are generated for single-hidden-layer feedforward neural networks with random hidden nodes (FNNRHN). It argues that the standard practice of drawing both weights and biases uniformly from the same interval [−u,u] is wasteful, because in low-dimensional problems most sigmoid inflection points fall outside the input hypercube H, leaving only saturated (nearly flat) fragments inside H. The proposed fix is to draw each bias as b = −a^T x^*, where x^* is a point in H (e.g., a random training point), which forces h(x^*) = 0.5 and hence places the inflection hyperplane of that sigmoid inside H (Section 2, Eq. (6)). The paper further proposes generating each weight as a = 4 tan α, where α is drawn uniformly from [α_min, α_max], in order to obtain hidden units with uniformly distributed slope angles (Section 3, Eq. (9)). Algorithm 1 combines these ideas. The empirical section compares the standard method (SM), the proposed bias-placement method (PMu), and the version with angle-uniform weights (PMα) on two synthetic target functions, Eq. (1) and Eq. (10), for dimensions n = 2, 5, 10, reporting test RMSE versus number of hidden nodes.","tokens_in":6078,"tokens_out":4599,"duration_ms":46960,"significance":"The mathematical core of the paper is mostly sound and useful. The derivation of the inflection-point density in Eq. (4) is correct for one-dimensional inputs, the bias rule in Eq. (6) is a simple and elegant construction that indeed guarantees a point of the inflection hyperplane inside H, and Eq. (9) is a clean parameter-free transformation from a slope angle to a weight. These are concrete, implementable tools that could help practitioners generate more informative random hidden units. The paper also provides a full algorithm and reproducible-looking simulation code. However, the significance of the empirical claim is currently limited. The two target functions are both strongly oscillatory, so they favor the paper's premise that only steep, inflection-centered fragments are useful; the absence of smoother targets makes the generality of the improvement unclear. In addition, the hyperparameter selection in Section 4 is not described precisely enough to rule out test-set leakage. If the authors address these evaluation issues, the contribution would be a useful practical guideline for randomized neural-network training.","major_comments":[{"comment":"The central improvement rests on the premise, stated in Section 2, that 'the saturated fragments of these sigmoids ... are useless for modeling nonlinear TFs.' This is a heuristic, not a theorem. A saturated sigmoid still has a nonzero derivative and can contribute a slowly varying or nearly constant component to a linear combination, which may be useful for smooth or low-frequency targets. The experiments use only the two oscillatory functions g(x) = sin(20 exp x) x^2 and its multivariate additive version, Eq. (10), so they cannot detect whether forcing all inflection hyperplanes into H helps on smoother targets or whether it reduces the diversity of the basis. I ask the authors to test the method on at least one smooth/low-frequency target and to provide either a quantitative argument for why saturated fragments are useless or a more nuanced statement of when Eq. (6) is expected to help.","section":"Section 2 (Eq. (6) and the discussion of saturated sigmoids)"},{"comment":"The text says 'For each node number the optimal value of u or α_min was selected from u ∈ {1, 2, ..., 10, 20, 50, 100} and α_min ∈ {0°, 10°, ..., 80°}, respectively,' but it does not state whether this selection was made on the test set, a validation set, or by some other criterion. If the test set was used to choose the hyperparameters, the reported PMα curves are envelopes over a grid of hyperparameters and the improvement over SM may be substantially overstated. Please specify the selection procedure, and ideally report results for a fixed hyperparameter chosen by a proper validation split or via nested cross-validation.","section":"Section 4, Figure 5 (hyperparameter selection)"},{"comment":"For n > 1 the sigmoid h(x) = σ(a^T x + b) has an (n−1)-dimensional inflection hyperplane, not an isolated point, so the statement 'probability that χ belongs to H' as a function of n is conceptually unclear unless χ is defined as a random point on that hyperplane or as the vector of coordinate-wise ratios. The subsequent construction in Eq. (6) is clear, but the motivation via Figure 2 would benefit from an explicit definition of what is being plotted for n > 1.","section":"Section 2, right panel of Figure 2 (multivariate inflection points)"}],"minor_comments":[{"comment":"Several equation references are incorrect: biases are set by Eq. (6), not Eq. (8), and in the definitions of PMu and PMα the biases are again said to follow Eq. (8) when they should follow Eq. (6). Please correct these cross-references.","section":"Section 4, first paragraph and method definitions"},{"comment":"The orientation of the output matrix A is not fully specified: the loop generates a_{i,j} with i indexing nodes and j indexing inputs, but the bias formula uses a_i^T x_i^*. Please clarify whether columns or rows of A correspond to hidden nodes, or use a consistent indexing convention.","section":"Algorithm 1"},{"comment":"The derivation of f(χ) would be easier to follow if the support of χ and the change of variables b = −aχ were stated explicitly; in the current typesetting the integration limits for |χ| ≥ 1 are easy to misread.","section":"Section 2, Eq. (4)"},{"comment":"The abstract and introduction say the standard method draws weights and biases from 'the same fixed interval,' but the paper also cites work that optimizes u. Please clarify the scope of the critique so as not to conflate the fixed-interval choice with the wider class of tuned uniform methods.","section":"Section 1"}],"recommendation":"major_revision","confidential_remarks":"The paper's contribution is incremental but potentially useful for practitioners. The main concern is whether the empirical evidence supports the claimed advantage: only two oscillatory synthetic targets are used, and the hyperparameter selection protocol for Figure 5 is ambiguous. A revision that adds a validation-split description and broader experiments, including smooth targets, would substantially strengthen the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take on arXiv:1908.05864. The useful core is a simple observation: with standard uniform generation of weights and biases, most scalar sigmoid inflection points fall outside the input interval (only 25% chance of being inside for n=1), so the steep, useful fragments of many hidden units are wasted. The proposed fix, b = −a^T x*, puts the inflection hyperplane exactly at a point in H. That is correct by construction, and the slope-angle alternative in Eq. (9) is a genuine new idea.\n\nWhat the paper does well: the math is right, the construction is simple, and the demonstration on the wiggly one-dimensional target is illustrative. The multivariate simulations are a start.\n\nWhere it is soft: the premise that saturated sigmoid fragments are useless is asserted, not established. If saturated units can still provide offsets or low-frequency corrections, forcing every inflection point inside H might reduce basis diversity. The paper's two synthetic targets are both high-frequency; that does not test the premise on smooth functions. Second, the hyperparameter selection in Sec. 4 is ambiguous: 'optimal u or alpha_min selected from a grid' is reported, but not on what data the selection was made. If it is the test set, the curves are envelopes and the improvement is overstated. Third, the multivariate claim is imprecise: in n-D the inflection set is a hyperplane, not a point, and the probability graph in Fig. 2 is derived for a scalar ratio. The extension to n ≥ 2 is not justified.\n\nFinally, the relation to the author's own previous work [10,11] is not clearly delineated. The bias rule appears to be a simple version of that earlier work, while the uniform-slope-angle part is the genuinely new piece.\n\nOverall, this is a coherent, honest paper with a real nugget, but the empirical evidence is too thin to support the central claim that forcing inflection points into the box helps in general. A proper revision with more datasets, a clear validation protocol, and error bars would make it worth publishing.\n\nMy recommendation: send it to a serious referee, but expect that the empirical claims need substantial strengthening. For my own citations, I would hold off until the follow-up.","headline":"A correct and simple bias-placement trick for random hidden nodes, with a new slope-angle parameterization, let down by a thin empirical evaluation and ambiguous hyperparameter selection.","tokens_in":6597,"tokens_out":5091,"would_cite":false,"duration_ms":49286,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"Placing each sigmoid's inflection point inside the input hypercube, via $b = -a^T x^*$, and drawing slope angles uniformly ($a = 4\\tan\\alpha$) improves random-hidden-node networks and cuts overfitting.","keywords":["feedforward neural networks","random hidden nodes","randomized learning","sigmoid activation","inflection point placement","uniform slope angles","input hypercube","bias generation"],"falsifier":"Take a low-dimensional regression task and build two random-hidden-node sets with identical slope-angle distributions, one using $b=-a^T x^*$ with all $x^*$ inside $H$ and one adding a fixed large offset so all inflection surfaces miss $H$; if their test errors are nearly equal, the paper's premise that only in-cube inflection fragments contribute would be falsified.","tokens_in":5671,"feed_emoji":"🧠","tokens_out":8871,"duration_ms":78101,"temperature":0.7,"pith_summary":"The paper argues that the conventional recipe for random-hidden-node feedforward networks—drawing both weights and biases from the same uniform interval—wastes most hidden nodes, because the most useful part of each sigmoid, the steep region around its inflection point, usually lands outside the input hypercube. The paper's fix is to generate the bias from a chosen point $x^*$ in the input domain by setting $b = -a^T x^*$, which forces the sigmoid's inflection surface through that point. It also replaces uniform weights with weights $a = 4\\tan\\alpha$ where $\\alpha$ is drawn uniformly between a problem-dependent minimum and maximum angle, giving hidden nodes with uniformly distributed slope angles. A sympathetic reader would care because, if correct, the same-size random network becomes more accurate, needs fewer nodes, and suffers less overfitting at high node counts.","feed_headline":"Sigmoid inflection points inside the input cube give better fits","feed_subtitle":"Biases derived from data points plus uniform slope angles cut the node count.","key_machinery":"The carrying object is the sigmoid inflection point $\\chi$, defined by $h(\\chi)=0.5$, together with the identity $\\tan\\alpha = a/4$ between the weight $a$ and the slope angle $\\alpha$ at that point. The standard method's deficiency is quantified by the distribution of $\\chi = -b/a$: under uniform $a,b$, only a quarter of one-dimensional sigmoids have their steepest fragment inside the input interval, and the fraction inside the hypercube grows with dimension (roughly 46% for $n=2$ and above 90% for $n\\ge 7$). The proposed method uses the bias rule $b=-a^T x^*$ with $x^* \\in H$ (a random point of $H$, a random training point, or a cluster prototype) to move every inflection surface into the input hypercube, and uses $a=4\\tan\\alpha$ with uniform $\\alpha$ to spread slope angles evenly.","core_discovery":"On the paper's own terms, the central discovery is that bias and weight play different roles in a random-hidden-node sigmoid and should therefore be generated differently. With standard draws $a,b \\sim U(-u,u)$, the inflection point obeys $\\chi = -b/a$, and the paper derives its density: $f(\\chi)=1/4$ for $|\\chi| < 1$ and $f(\\chi)=1/(4\\chi^2)$ for $|\\chi| \\ge 1$, so for one input only 25% of sigmoids have their steepest fragment in the unit interval. Choosing any $x^*$ inside the input domain and setting $b = -a^T x^*$ puts a sigmoid's inflection surface directly at $x^*$, so every hidden node can be made locally flexible where the target fluctuates. Independently, because the sigmoid's slope at its inflection point satisfies $\\tan\\alpha = a/4$, generating $\\alpha \\sim U(\\alpha_{\\min}, \\alpha_{\\max})$ and setting $a = 4\\tan\\alpha$ produces uniformly distributed slope angles, which the experiments link to reduced overfitting.","pith_inferences":["The bias-placement rule should transfer to other sigmoidal or threshold-like activations, where one redefines $x^*$ as the point at which the activation passes through its central value and sets the shift accordingly.","Uniform slope angles offer a principled regularisation dial: capping $\\alpha_{\\max}$ by the steepest slope the target function actually exhibits would replace the trial-and-error choice of $u$ with an interpretable geometric bound.","Drawing $x^*$ from the most recent data would let random-hidden-node networks track nonstationary inputs without retraining output weights, a streaming adaptation the paper does not discuss.","The paper's own probability calculation predicts that the improvement over standard generation should shrink as input dimension grows; testing the method on low-dimensional benchmarks is therefore the most discriminating next experiment."],"forward_implications":["Standard uniform generation is wasteful in low-dimensional problems, where only about a quarter of sigmoids have their steepest fragments in the input interval; the proposed bias rule makes the useful fragment present for every node.","Selecting $x^*$ as a random training point instead of a uniformly random point of $H$ distributes the hidden nodes according to data density, and selecting cluster prototypes gives a data-adaptive placement.","Uniform slope angles from $U(\\alpha_{\\min}, \\alpha_{\\max})$ prevent the concentration of near-vertical sigmoids that occurs with wide uniform weight ranges, which the simulations associate with overfitting.","In the reported experiments, the angle-based version $PM_\\alpha$ attains the lowest mean test errors for $n=2,5,10$, and for $n=5$ and $n=10$ its error keeps decreasing as node count grows while the standard methods' error rises.","For the single-variable target, the proposed method reached a similar RMSE with 35 hidden nodes while the standard method needed more than 60."],"supporting_citations":[{"why":"It supplies the convex least-squares output-weighting procedure that makes random-hidden-node networks fast and is used throughout the experiments.","marker":"[1]"},{"why":"It provides the universal-approximation guarantee for random parameters drawn from a symmetric interval, the baseline guarantee the proposed generation scheme must preserve.","marker":"[2]"},{"why":"It identifies the choice of interval and distribution for random parameters as the most important open problem in randomized learning, which motivates the paper's investigation.","marker":"[3]"},{"why":"It surveys neural networks with random weights and frames parameter generation as a central research gap.","marker":"[4]"},{"why":"It documents the un-justified standard practice of drawing weights and biases from $[-1,1]$, the practice the paper diagnoses and replaces.","marker":"[5]"},{"why":"It is the author's earlier method for generating random weights and biases in a data-dependent way, the direct precursor whose drawback the new bias rule addresses.","marker":"[10]"},{"why":"It is an earlier proposal for appropriate generation of random parameters and provides the near context for the improved scheme.","marker":"[11]"}],"fun_headline_variants":["Bias sets sigmoid inflection inside input hypercube","Uniform slope angles instead of uniform weights","Place inflection at data points for better fits","Sigmoid steepest fragment goes where it's needed","Uniform slope angles cut hidden node count"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that only each sigmoid's steepest fragments around its inflection level $h=0.5$ are useful for modeling target-function nonlinearity, so saturated pieces inside the input cube cannot compensate; the simulations also presume that the tuning bounds $u$ or $\\alpha_{\\min}$ are chosen on validation data only, without leaking test information.","fun_headline_variants_meta":{"raw":{"variants":["Bias sets sigmoid inflection inside input hypercube","Uniform slope angles instead of uniform weights","Place inflection at data points for better fits","Sigmoid steepest fragment goes where it's needed","Uniform slope angles cut hidden node count"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000422,"raw_usage":{"total_tokens":2123,"prompt_tokens":858,"completion_tokens":1265,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":474,"completion_tokens_details":{"reasoning_tokens":1196}},"tokens_in":474,"tokens_out":1265,"duration_ms":9526,"temperature":1.0,"reasoning_tokens":1196,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:03:21.031151+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a low-dimensional regression task and build two random-hidden-node sets with identical slope-angle distributions, one using $b=-a^T x^*$ with all $x^*$ inside $H$ and one adding a fixed large offset so all inflection surfaces miss $H$; if their test errors are nearly equal, the paper's premise that only in-cube inflection fragments contribute would be falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the universal-approximation guarantee for random parameters drawn from a symmetric interval, the baseline guarantee the proposed generation scheme must preserve."},{"cited_title":"Zhang, P","cited_arxiv_id":null,"evidence_quote":"It identifies the choice of interval and distribution for random parameters as the most important open problem in randomized learning, which motivates the paper's investigation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It surveys neural networks with random weights and frames parameter generation as a central research gap."},{"cited_title":"Scardapane, D","cited_arxiv_id":null,"evidence_quote":"It documents the un-justified standard practice of drawing weights and biases from $[-1,1]$, the practice the paper diagnoses and replaces."},{"cited_title":"Dudek, Generating random weights and biases in feedforwar d neural networks with random hidden nodes, Information Sciences 481 (20 19) 33–","cited_arxiv_id":null,"evidence_quote":"It is the author's earlier method for generating random weights and biases in a data-dependent way, the direct precursor whose drawback the new bias rule addresses."},{"cited_title":"Dudek, Improving randomized learning of feedforward neur al networks by appropriate generation of random parameters, in: Advances in Compu- tational Intelligence","cited_arxiv_id":null,"evidence_quote":"It is an earlier proposal for appropriate generation of random parameters and provides the near context for the improved scheme."}],"review_version":1}