{"id":"0eabd905-78f4-40d4-ba56-c8ceee5a90ce","arxiv_id":"2412.20851","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Rectified sigmoid (hard sigmoid) activation is reported to cut PINN solution errors by about an order of magnitude on two ODE benchmarks, but the result may be an interpolation artifact because the paper never discloses the source of the initialization data.","lead":"A research team proposes using the rectified sigmoid, a ramp activation that is just the known hard sigmoid, in single-layer physics-informed neural networks and reports large accuracy gains on three ODE problems. The gains are in the tables, but the paper never reveals where the data used to initialize the networks comes from, so the networks may be interpolating a known solution rather than solving the equations.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1 consumes solution values {u_k}; if these come from the odeint reference solver, Section 3.2's order-of-magnitude comparison measures interpolation, not PINN solving.","rationale":"The reader's weakest assumption is precisely the one I would stress-test: Algorithm 1 requires solution values {u_k} before any solving happens, and the paper never discloses their provenance. I agree that this is the single most load-bearing point, because Table 1 — the only evidence for the 'at least an order of magnitude' claim — reports errors after PIDD initialization alone. If {u_k} come from odeint, the results are interpolation errors, not PINN solution errors, and the comparison with sigmoid is a comparison of interpolation bases. The Section 2 derivation itself is internally consistent, and Algorithm 2's NbN training is a genuine unsupervised forward method, so the paper is not a clear reject; however, its headline numerical claim is currently unsupported. I would keep the reader's CONDITIONAL verdict: the condition should be that the authors state exactly how {u_k} are generated and rerun the PIDD experiments without oracle data, or the 'order of magnitude' claim should be withdrawn. No new concern beyond the reader's was identified, so the verdict is unchanged.","tokens_in":9162,"tokens_out":8085,"duration_ms":85928,"concrete_test":"Re-run the harmonic oscillator and Lorenz experiments in Section 3.2 with Algorithm 1, but generate {u_k} by a first-order explicit Euler integrator starting from u(0) on the same 20,000-point grid, keeping everything else identical. Compare the resulting relative L2 errors against Table 1. If the errors jump from ~1e-6 to ~1e-2 or worse, the low errors in the paper come from feeding the odeint reference solution into the initialization, not from the rectified sigmoid. If the errors remain ~1e-6, the method is a genuine forward solver and the concern is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central numerical claim in Section 3.2 — that replacing sigmoid with Re-sigma reduces relative L2 error by at least an order of magnitude — rests entirely on Table 1, which reports errors after PIDD initialization only. Algorithm 1 begins with 'Data: {u_k}_{k=0}^{N-1}' and sets W_k^(2) = -Delta x * N_l[u_k, x_k] / Delta zeta (Eq. 8). This requires solution values at every collocation point before the network is constructed. The paper never states where these values come from; the only solver mentioned in Section 3 is scipy's odeint, used to define the 'reference solutions'. If the {u_k} fed to Algorithm 1 are those odeint outputs, then the network is effectively a piecewise-linear interpolant/quadrature of the reference trajectory, and the error in Table 1 measures how well Re-sigma basis functions reproduce the reference data, not how well a PINN solves the ODE. The comparison with sigmoid then compares two interpolation bases rather than two physics-informed solvers. This is the load-bearing premise: without a forward source for {u_k}, the 'order-of-magnitude' improvement does not support the paper's central claim. The NbN results in Table 2 are not subject to this particular circularity because Algorithm 2 has 'Data: —', but for Lorenz the Re-sigma advantage there is only about 1.3x, so the strong 'any given value' claim rests on the PIDD numbers whose provenance is unstated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a rectified sigmoid activation function Re-sigma (Eq. 4) for single-hidden-layer physics-informed neural networks solving ODEs, together with a physics-informed data-driven (PIDD) initialization (Algorithm 1) and a neuron-by-neuron (NbN) training procedure (Algorithm 2) adapted from previous work. The main technical claim is that, because Re-sigma has compact support and Delta-zeta = 1, each neuron affects the derivative only at its own collocation point, so the output weights are determined exactly by the ODE residual, as shown in Eqs. (6)-(8). The numerical experiments in Section 3 compare relative L2 errors of networks with sigmoid and Re-sigma activations on a harmonic oscillator, a relativistic slingshot, and a Lorenz system, and Section 3.2 claims an order-of-magnitude error reduction for any given value. The central unresolved issue is the provenance of the solution values {u_k} consumed by Algorithm 1, which the paper never states.","tokens_in":9508,"tokens_out":5758,"duration_ms":52197,"significance":"If the central claim were fully established, the paper would offer a practically useful, almost parameter-free way to obtain accurate ODE solutions from tiny single-hidden-layer networks, with a simple and elegant derivative construction. The Section 2 derivation is correct as far as it goes: the compact-support argument makes Eq. (8) exact at the grid points, and the explicit algorithms make the method easy to reproduce. However, the claimed superiority over the sigmoid activation is only as strong as the source of the input data in Algorithm 1; if {u_k} are the reference solver values, the reported errors are interpolation errors, not physics-informed solution errors. The paper also does not quantify uncertainty, but because the algorithms are deterministic this is less important than the data-source issue.","major_comments":[{"comment":"The PIDD initialization in Algorithm 1 requires the solution values {u_k} at every collocation point, and Eq. (8) sets W_m^(2) = -Delta x N_l[u_m, x_m]/Delta zeta from those values. The paper never states where these values come from; in Section 3 the only numerical reference mentioned is scipy's odeint solver, used for the 'reference solutions'. If {u_k} are the odeint outputs, then the networks in Table 1 are constructed by interpolation of the reference trajectory, not by solving the ODE from the initial condition alone, and the comparison with the sigmoid activation is a comparison of two interpolation bases. Because Section 3.2's order-of-magnitude claim rests entirely on Table 1, this missing provenance is load-bearing. Please state the source of {u_k} explicitly, or re-run the PIDD experiments using only the initial condition and the ODE residual.","section":"Algorithm 1; Section 3.2; Eq. (8)"},{"comment":"The claim in Section 3.2 that replacing sigmoid with rectified sigmoid 'results in a reduction of the relative error by at least an order of magnitude for any given value' is not supported by the NbN results: in Table 2, for the Lorenz system the Re-sigma relative errors are only about 1.3 times smaller than the sigma errors (e.g., 2.63e-3 vs 3.47e-3 for x). If the claim is meant to cover only the PIDD initialization, the text should say so; if it is meant to cover the proposed method generally, the statement needs to be revised to match Table 2.","section":"Section 3.2 vs. Section 3.3, Tables 1 and 2"},{"comment":"The observation that Re-sigma's relative L2 errors for the harmonic oscillator (3.88e-6 and 2.63e-6) are close to the odeint-vs-exact errors (1.03e-6 and 1.04e-6) is consistent with the interpretation that the Re-sigma network is reproducing the odeint reference solution to the accuracy of that solver. The paper should state whether the reported errors therefore measure the network's fit to the reference data, and should discuss how this relates to the claim that the network 'solves' the ODE.","section":"Section 3.2, harmonic oscillator paragraph"}],"minor_comments":[{"comment":"The system in Section 3.1.3 and in Tables 1 and 2 is called 'Lorentz system'; the standard name is 'Lorenz system' (also used in Eq. (13)).","section":"Throughout"},{"comment":"There is a typo: 'derivate' should be 'derivative', and the phrase 'Pay attention Re-sigma'(0) = 1/2' needs a comma or semicolon.","section":"Section 2, Eq. (6)"},{"comment":"The subscript l is used in Eq. (8) (N_l[u_m, x_m]) but omitted in Algorithm 1's W_k^(2) update; make the component index consistent.","section":"Algorithm 1 vs Eq. (8)"},{"comment":"The initialization in Algorithm 2 uses N_l[u(0), x_k] but the notation u(0) is ambiguous for a system: it should be stated whether this means the constant initial vector evaluated at every collocation point, and how this differs from the correction term b0 used in Algorithm 1.","section":"Algorithm 2"},{"comment":"The text says the absolute errors are 'significantly lower' than those of vanilla PINN, but no definition, settings, or results for vanilla PINN are given; either cite the specific comparison in [21] or add the missing details.","section":"Section 3.3, Figure 1"}],"recommendation":"major_revision","confidential_remarks":"The missing provenance of the input {u_k} in Algorithm 1 is the crux. If those values come from the reference solver, the paper's main numerical claim needs substantial reframing. The Section 2 derivation itself appears sound and the idea may still be useful, but the manuscript in its current form does not establish the claimed PINN accuracy improvement."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the core trick is real but smaller than claimed. Replacing sigmoid with the clamped linear ramp (hard sigmoid, Eq. 4) gives each hidden neuron a compact-support derivative, so with Δζ=1 the PIDD weights can be chosen so each neuron matches the ODE residual exactly at its own collocation point. That part of the derivation (Eqs. 6–8) is correct, and the NbN gradient-free update that follows is a reasonable way to iterate the weights. If you work on tiny single-layer PINNs, this is a useful trick to know about.\n\nThe problems are in what the experiments actually demonstrate. Algorithm 1 consumes the full solution values {u_k} on the grid. The paper never says where these come from; the only solver mentioned is scipy's odeint, which also provides the reference for the error. If {u_k} are odeint output, then the PIDD network is a piecewise-linear interpolant of the reference solution, and Table 1 measures interpolation error, not physics-informed solving. The comparison against sigmoid is then a comparison of two interpolation bases, not two solvers. That doesn't make the numbers useless, but it makes the phrase 'solving physical problems' misleading. The NbN results are not circular (Algorithm 2 needs only u(0)), but they undercut the 'order of magnitude for any given value' claim: on Lorenz, Re-σ beats sigmoid by only ~1.3x, not 10x. So the strong statement in Section 3.2 is simply false as written.\n\nThere are two smaller issues. One, Eq. (4) is the standard hard sigmoid, known for decades in the binary neural network literature; presenting it as a new activation function is not right. Two, there are no error bars, single runs, and no code, so the reported improvements are fragile.\n\nWorth a serious referee? Yes, if the authors are pushed to state exactly how {u_k} are generated, soften the overgeneralized claim, and add statistical rigor. The derivation is sound and the trick has potential for lightweight surrogates, but as it stands the paper's headline claim rests on a comparison whose data provenance is undisclosed.\n\nRecommendation: send to peer review with a request for major revision, mainly to clarify the role of the reference data and to rescope the claim.","headline":"A correct and neat derivation of a piecewise-linear activation trick for small PINNs, but the headline accuracy claim rests on interpolation with undisclosed reference data and is broader than the Lorenz numbers support.","tokens_in":10044,"tokens_out":4269,"would_cite":false,"duration_ms":42193,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","65L05"],"pacs":[],"model":"deepseek-v4-flash","headline":"A rectified sigmoid makes single-hidden-layer PINNs an order of magnitude more accurate on ODEs.","keywords":["rectified sigmoid","physics-informed neural networks","single hidden layer","ordinary differential equations","PIDD initialization","neuron-by-neuron training","harmonic oscillator","Lorenz system"],"falsifier":"Take a new ODE with a known exact solution, initialize the Re-$\\sigma$ network without supplying any grid values $\\{u_k\\}$ — setting $W^{(2)}_k$ only from the ODE residual and the initial condition — and compare against the sigmoid network. If the Re-$\\sigma$ network no longer beats sigmoid by an order of magnitude, the reported gain depends on the precomputed solution points.","tokens_in":8947,"feed_emoji":"📉","tokens_out":8499,"duration_ms":79068,"temperature":0.7,"pith_summary":"This paper tries to establish that a small, explicit change to the activation function — replacing the smooth sigmoid with a piecewise-linear \"rectified sigmoid\" — improves the accuracy of single-hidden-layer physics-informed neural networks on ordinary differential equations by at least an order of magnitude. The reported gains appear in three very different test systems: a linear harmonic oscillator, a relativistic electron slingshot, and the chaotic Lorenz system. If the claim holds, accurate physics-informed solving no longer needs deep networks or gradient-based training; a closed-form initialization followed by a few gradient-free fitting passes would suffice. The proposed initialization is \"physics-informed data-driven\" (PIDD), and the fitting method is \"neuron-by-neuron\" (NbN), both adapted from the sigmoid version of the method.","feed_headline":"Swap the activation, gain tenfold accuracy on ODEs","feed_subtitle":"A piecewise-linear cap lets one hidden layer rival standard ODE solvers on three very different systems.","key_machinery":"The load-bearing object is the rectified sigmoid activation, defined piecewise as $\\mathrm{Re}\\text{-}\\sigma(x)=0$ for $x<-1$, $\\tfrac12(x+1)$ for $-1\\le x\\le 1$, and $1$ for $x>1$, which is a sum of two ReLU functions: $\\mathrm{Re}\\text{-}\\sigma(x)=\\tfrac12[\\mathrm{ReLU}(x+1)-\\mathrm{ReLU}(x-1)]$. Because its derivative is the box function with $\\mathrm{Re}\\text{-}\\sigma'(0)=1/2$ and with the chosen spacing $\\Delta\\zeta=1$ the derivative of one neuron does not overlap the neighbouring neuron, the initialization formula collapses to $W^{(2)}_k = -\\Delta x\\, N_l[u_k,x_k]/\\Delta\\zeta$, a direct closed-form expression for the output weights from the ODE residual at grid points. This converts the physics equations into an explicit, gradient-free construction of the network.","core_discovery":"The central claim is that substituting the sigmoid activation with the rectified sigmoid $\\mathrm{Re}\\text{-}\\sigma(x) = 0$ for $x<-1$, $\\tfrac12(x+1)$ for $-1\\le x\\le 1$, and $1$ for $x>1$ reduces the relative $L^2$ error of the predicted ODE solution by at least an order of magnitude for any given value, after the same initialization and training procedures. The paper reports this for PIDD initialization and for NbN training (three epochs), on the harmonic oscillator, the relativistic slingshot, and the Lorenz system. In the harmonic-oscillator case the Re-$\\sigma$ network's error is close to the error of the standard high-precision reference solver itself.","pith_inferences":["The paper compares Re-$\\sigma$ only against the sigmoid; a direct comparison with plain ReLU under the same PIDD initialization would isolate whether the benefit comes from the rectified shape or from the smooth central slope, an experiment the paper does not run.","The 'any given value' strength of the claim rests on three systems at two widths; a width and parameter sweep would show whether the gap persists for very small networks and for PDEs.","Because the output weights are set by the residual formula, the scheme could be used for fast parameter sweeps or surrogate modeling of a family of ODEs by re-initializing with new residuals, an application left implicit.","If the solution values consumed by the PIDD algorithm come from a reference solver, the reported errors measure interpolation of that solver's output; a natural test is to run the same initialization with the grid values withheld and watch whether the gap survives."],"forward_implications":["Single-hidden-layer networks with the rectified sigmoid reach errors near $10^{-6}$ on the harmonic oscillator, close to the reference solver's own error in the reported setup.","The reported order-of-magnitude gap over the sigmoid appears both after initialization alone (PIDD) and after the gradient-free NbN training, and across linear, nonlinear, and chaotic systems.","Training stays cheap: the PIDD runs in milliseconds and the NbN fitting in a few minutes in the reported experiments, with three epochs sufficient.","Because the same algorithms transfer from sigmoid to Re-$\\sigma$ with only the spacing constant $\\Delta\\zeta$ changed, the method applies directly to the earlier single-hidden-layer toolbox."],"supporting_citations":[{"why":"Supplies the PIDD initialization and neuron-by-neuron training algorithms that the paper adapts to the rectified sigmoid, and the sigmoid baseline it compares against.","marker":"[21]"},{"why":"Defines the physics-informed neural network framework that the network architecture and the physics residual loss build on.","marker":"[3]"},{"why":"Provides the survey context for sigmoid and ReLU activations used to justify the rectified construction and Euler-method suitability.","marker":"[22]"},{"why":"Gives the physical model of the relativistic slingshot used as one of the three benchmark problems.","marker":"[25]"},{"why":"Supplies the classical Lorenz system equations and parameters used as the chaotic benchmark.","marker":"[26]"}],"fun_headline_variants":["Rectified sigmoid boosts PINN accuracy tenfold on ODEs","One activation swap gives 10x better PINN solutions","PINNs with rectified sigmoid beat standard by 10x on ODEs","A simple activation change makes PINNs 10x more accurate","Rectified sigmoid: physics-informed nets get 10x precision"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The initialization algorithm needs the solution values $\\{u_k\\}$ at the grid points before training, and the paper never states where they come from; if they come from the same reference solver used to measure error, the reported accuracy may reflect interpolation of that solver's output rather than physics-informed prediction.","fun_headline_variants_meta":{"raw":{"variants":["Rectified sigmoid boosts PINN accuracy tenfold on ODEs","One activation swap gives 10x better PINN solutions","PINNs with rectified sigmoid beat standard by 10x on ODEs","A simple activation change makes PINNs 10x more accurate","Rectified sigmoid: physics-informed nets get 10x precision"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000674,"raw_usage":{"total_tokens":2994,"prompt_tokens":800,"completion_tokens":2194,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":416,"completion_tokens_details":{"reasoning_tokens":2101}},"tokens_in":416,"tokens_out":2194,"duration_ms":16307,"temperature":1.0,"reasoning_tokens":2101,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:09:23.740506+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a new ODE with a known exact solution, initialize the Re-$\\sigma$ network without supplying any grid values $\\{u_k\\}$ — setting $W^{(2)}_k$ only from the ODE residual and the initial condition — and compare against the sigmoid network. If the Re-$\\sigma$ network no longer beats sigmoid by an order of magnitude, the reported gain depends on the precomputed solution points.","supporting_citations":[{"cited_title":"Are Two Hidden Layers Still Enough for the Physics-Informed Neural Networks?","cited_arxiv_id":"2412.19235","evidence_quote":"Supplies the PIDD initialization and neuron-by-neuron training algorithms that the paper adapts to the rectified sigmoid, and the sigmoid baseline it compares against."},{"cited_title":"Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,","cited_arxiv_id":null,"evidence_quote":"Defines the physics-informed neural network framework that the network architecture and the physics residual loss build on."},{"cited_title":"Relativistic slingshot: A source for single circularly polarized attosecond x-ray pulses,","cited_arxiv_id":null,"evidence_quote":"Gives the physical model of the relativistic slingshot used as one of the three benchmark problems."},{"cited_title":"Deterministic nonperiodic flow,","cited_arxiv_id":null,"evidence_quote":"Supplies the classical Lorenz system equations and parameters used as the chaotic benchmark."}],"review_version":1}