{"id":"4f3d1db9-5696-4462-a0b6-cf7070fe7a9e","arxiv_id":"2507.22045","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"Legendre-polynomial weight parameterization lowers training cost and improves stability in continuous-time network surrogates, but the reported accuracy advantage conflicts with the paper's own error table.","lead":"This paper tests whether representing time-varying neural network weights as Legendre polynomials instead of plain monomials makes continuous-time models like neural ODEs and ResNets train faster and more stably for simulating physics. It reports that Legendre bases can cut function evaluations by an order of magnitude, but a key accuracy table in the paper contradicts its own headline claim.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 5 and Figure 6 contradict each other on DCR neural-ODE accuracy, so the 'comparable or better accuracy' claim is not supported as written.","rationale":"The paper's headline is a comparative claim: Legendre is at least as accurate as monomial and costs less. The computational-cost half is supported by the function-evaluation counts in Table 5, though those counts become less meaningful if Legendre stops early at a worse error. The accuracy half is decided by the error columns of Table 5, and the DCR row of that table contradicts both the surrounding prose and Figure 6. This is not a modeling assumption or a difference of opinion; it is an internal inconsistency in the primary evidence. A reader cannot verify whether Legendre is better, equal, or worse on DCR without the seed logs. Because DCR is one of only three benchmarks, the claim 'comparable to or better than monomial' collapses if the table is right, and the reporting is unreliable if the figure is right. The reader's weakest_assumption concerned identical hyperparameters being unfair to monomials; that is a real secondary issue, but it would only weaken the comparison after the numbers are reconciled. The table/figure conflict is more load-bearing because it must be resolved before any cross-basis comparison can be interpreted. I therefore keep the reader's CONDITIONAL verdict: the paper should not be accepted until the raw data are made available and the DCR numbers are reconciled; if the raw data confirm Table 5, the central accuracy claim should be rejected or substantially narrowed.","tokens_in":21742,"tokens_out":7456,"duration_ms":87926,"concrete_test":"Extract the raw per-seed training and validation histories for the DCR neural ODE runs behind Table 5 (five seeds, third-order Legendre and monomial, optimize-then-discretize with DOPRI5) and recompute the means, standard deviations, and function-evaluation counts from those logs, overlaying all five seeds on the Figure 6 axes. If the recomputed numbers match Table 5, Legendre is worse on DCR and the central accuracy claim must be narrowed or the benchmark dropped. If the recomputed numbers match the text and Figure 6, the table was mislabeled; correct it and rerun the same reconciliation check for ELM and CDR before acceptance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.3's central accuracy claim rests on Table 5 and Figure 6, but for DCR these two exhibits disagree. Table 5 reports DCR mean training/validation errors of 0.282/0.281 for Legendre and 0.036/0.037 for monomial, with Legendre standard deviations of 0.617/0.607 over five seeds. The text and Figure 6 state the opposite: the Legendre model reaches a validation loss of about 0.28, while the monomial case fails to drop below 0.6. If Table 5 is correct, Legendre is roughly 7.6x worse in mean error on DCR and far less stable across seeds, directly contradicting 'comparable or better accuracy' and 'more stable training dynamics.' If Figure 6 and the text are correct, Table 5 is mislabeled or contains erroneous entries. Either way, the submitted evidence does not currently establish the accuracy half of the headline claim for one of the three benchmarks, and the discrepancy cannot be resolved without seed-level data.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript studies polynomial weight-parameterization for continuous-time deep networks: it expands time-dependent weights in monomial and Legendre bases and applies both to ResNet-style (discretize-then-optimize) and neural-ODE (optimize-then-discretize) training on three surrogate benchmarks (ELM, CDR, DCR). The main claim is that Legendre bases give more stable training, lower computational cost, and accuracy comparable to or better than monomial or unconstrained weights. The function-evaluation reductions in Table 5 support the cost part of this claim, but the accuracy and stability parts are not consistently supported by the paper's own exhibits, and the DCR neural-ODE results in Table 5 and Figure 6 are mutually inconsistent.","tokens_in":21889,"tokens_out":9492,"duration_ms":96365,"significance":"If the reported Legendre advantages held, the paper would provide a simple, useful default choice for time-dependent weight parameterization in neural-ODE surrogates. The manuscript has a clear experimental design, and the efficiency evidence is strong: across all three datasets, Legendre reduces the number of function evaluations by roughly an order of magnitude (Table 5). However, the accuracy and stability claims are currently not established: for CDR and DCR, Table 5 shows monomial with lower mean error and lower variance, and the DCR figure contradicts the DCR table. The paper provides neither code nor seed-level data, so the reported numbers cannot be checked externally. The conditioning explanation in Section 4.3 is plausible but is asserted rather than demonstrated. With corrected exhibits, per-seed data, and a fairer hyperparameter protocol, the comparison could be made convincing.","major_comments":[{"comment":"The DCR rows of Table 5 and the DCR panel of Figure 6 report opposite accuracy orderings. Table 5 gives Legendre mean training/validation errors of 0.282/0.281 with standard deviations 0.617/0.607, and monomial errors of 0.036/0.037 with standard deviations 0.055/0.056, so monomial is about 7.6x more accurate and far more stable across the five seeds. Figure 6 and the accompanying text instead say the Legendre model reaches a validation loss near 0.28 while monomial fails to drop below 0.6. Both cannot be correct. The abstract's 'accuracy comparable to or better' and 'more stable training dynamics' claims rest on these exhibits; moreover, Table 5 does not report the non-parameterized neural-ODE error values needed to quantify the 'comparable to or better than unconstrained' part of the claim. Please reconcile the exhibits and provide seed-level curves and non-parameterized numeric errors.","section":"§5.3, Table 5 and Figure 6"},{"comment":"The comparison uses identical hyperparameters for monomial and Legendre bases (learning rate 0.001, batch size 32, up to 1000 epochs, ADAM and GNvpro), so the paper's attribution of monomial failure to Vandermonde ill-conditioning is not established. A basis that requires a smaller learning rate or more epochs would look unfairly bad under fixed settings. Please either report a per-basis hyperparameter search, show that monomial still fails under well-tuned settings, or provide direct condition-number/spectral diagnostics for the two Vandermonde matrices in Eqs. (11) and (13). Without this, the observed Legendre advantage may be an optimization-protocol artifact rather than a property of the basis.","section":"§5.2–5.3 and §4.3"},{"comment":"For CDR, Table 5 reports Legendre mean training/validation errors of 282.2/326.4 versus monomial 264.9/297.9, with Legendre standard deviations about four times larger (104/92 versus 26/28). The text in §5.3 states only that 'Legendre remains the most efficient' and does not acknowledge that monomial achieves lower mean error and lower variance on this benchmark. This contradicts the 'accuracy comparable to or better' and 'more stable training dynamics' claims for a second dataset. Please report the seed-level results, investigate whether the large Legendre variance reflects failed runs, and either revise the accuracy/stability claims or provide an explanation (e.g., a different convergence tolerance).","section":"§5.3, Table 5 (CDR rows)"},{"comment":"The paper claims that Legendre parameterization 'introduces negligible overhead in terms of implementation or FLOPs,' but the only cost evidence provided is the number of function evaluations in Table 5. Function evaluations are not the same as FLOPs, wall-clock time, or memory, and the polynomial basis evaluation and the adjoint integration can change the per-evaluation cost. Please report measured runtime/FLOP/memory comparisons, or rephrase the efficiency claim to refer specifically to the reported function-evaluation counts.","section":"§5.3"}],"minor_comments":[{"comment":"The acronym DCR is used inconsistently: Section 1 calls it 'direct current resistivity,' while Section 5.1 labels it 'Diffusion-Convection.' Please make the acronym and model description consistent.","section":"§5.1 and §1"},{"comment":"The data availability section contains the duplicated sentence 'The CDR dataset can be downloaded here.' with no actual URL. Provide working links for all datasets and, ideally, the code used to produce Tables 2–5 and Figures 2–6.","section":"Data availability"},{"comment":"The caption of Table 2 refers to the ELM surrogate model, but the table contains rows for DCR and CDR as well; correct the caption and clarify the apparent '634' entry in the CDR non-parameterized ADAM row (T=1).","section":"Table 2 caption"},{"comment":"Equation (9) and the gradient formula in Section 3.2 contain notational problems: the displayed integral '∫ a(t)^T ∇θ f(...) a(t) dt' appears to have an extra 'a(t)' factor, and Eq. (8) writes '∂u(t)/∂θP(t)' as if it were a scalar derivative of a vector-valued function. Please use full Jacobian notation.","section":"§3.2 and Eq. (9)"},{"comment":"Typographical issues: 'Picard-Linderlof' should be 'Picard-Lindelöf,' and the caption of Figure 6 should state what the colored lines correspond to and explain why the legend order changes between panels.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper is currently not acceptable in its submitted form. The internal contradiction between Table 5 and Figure 6 is serious and must be resolved with per-seed data, and the fixed-hyperparameter comparison is a further weakness that a careful revision should address. If the authors can correct the exhibits, provide code/data, and either demonstrate the Legendre advantage under a fair tuning protocol or appropriately narrow the claims, the paper could become publishable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: the efficiency story is real, the accuracy story is not as written. The paper compares monomial vs Legendre polynomial parameterization of time-varying weights in ResNets, Hamiltonian networks, and neural ODEs on three surrogate modeling tasks, under both discretize-then-optimize and optimize-then-discretize training. The side-by-side comparison across architectures and training paradigms is new, even though the parameterization itself is essentially NANODE's. The computational savings are substantial: in neural ODEs, Legendre cuts function evaluations by about an order of magnitude at the same polynomial degree, on all three benchmarks. That is a practically useful result, and the paper is clearly written.\n\nThe soft spot is load-bearing. Table 5, the paper's only seed-level statistics, says Legendre is worse than monomial on CDR (validation 326 vs 298) and far worse on DCR (validation 0.281 vs 0.037, std 0.607 vs 0.056). But the text and Figure 6 claim the opposite for DCR: Legendre reaches about 0.28 while monomial never goes below 0.6. These cannot both be right. Either the table is mislabeled or the figure is from a different configuration. As submitted, the headline claim of 'comparable or better accuracy' is unsupported for two of three benchmarks. The efficiency claim survives; the accuracy claim needs seed-level reconciliation.\n\nThere is also an incorrect statement in Section 4.2: constant-weight neural ODEs do not need time parameterization to compute gradients; the adjoint method in Chen et al. handles that fine. It is not central to the experiments, but it is wrong. A smaller issue: the ResNet experiments fix the same learning rate, batch size, and epoch count for both bases, so the monomial failure, while plausibly due to Vandermonde conditioning, is not checked against different optimization settings. And the ResNet figures lack error bars.\n\nWho is this for? Practitioners in scientific ML who want a cheaper default weighting scheme in continuous-time surrogates. The efficiency result is probably real and worth adopting. But the paper needs a major revision: reconcile Table 5 with the text/figure, release seed-level curves, and correct the Section 4.2 claim. I would send it to peer review because the comparison is useful and the efficiency claim is likely correct, despite the current reporting problems.","headline":"Useful efficiency comparison, but the neural-ODE accuracy claim is contradicted by the paper's own Table 5 on two benchmarks.","tokens_in":22499,"tokens_out":3547,"would_cite":true,"duration_ms":34906,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","65F35","65L06"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that Legendre polynomial bases are the right choice for parameterizing time-dependent weights in continuous-time neural networks, because their orthogonality keeps training stable and cheap while preserving accuracy.","keywords":["neural ODEs","weight parameterization","surrogate modeling","residual neural networks","Legendre polynomials","monomial basis","orthogonal polynomial basis","discretize-then-optimize"],"falsifier":"Rerun the DCR neural-ODE comparison with monomial parameterization using per-basis tuned learning rates, schedules, and a rescaled time interval, alongside a direct measurement of the monomial and Legendre Vandermonde condition numbers on the actual time grid; if monomial convergence and function-evaluation counts become comparable to Legendre's, the claimed conditioning advantage is not decisive.","tokens_in":21446,"feed_emoji":"🧮","tokens_out":7694,"duration_ms":82834,"temperature":0.7,"pith_summary":"Continuous-time neural networks — neural ODEs and residual networks — are attractive surrogates for expensive physical simulators, but their time-varying weights are costly and unstable to train. This paper asks whether constraining those weights to a low-dimensional polynomial subspace makes training easier, and which polynomial basis should be used. The central claim is that an orthogonal basis matters: Legendre-parameterized weights train more stably, need fewer function evaluations, and match or beat both monomial parameterizations and unconstrained weights in accuracy. This is tested in both discretize-then-optimize and optimize-then-discretize settings across three high-dimensional surrogate benchmarks. If correct, the practical message is to reach for orthogonal polynomial bases when parameterizing time-dependent network weights.","feed_headline":"Legendre weight bases beat monomials in continuous-time nets","feed_subtitle":"On three surrogate benchmarks, orthogonal bases match unconstrained weights with up to 8x fewer function evaluations.","key_machinery":"The central object is the polynomial weight parameterization $\\theta_P(t)=\\sum_{i=1}^d p_i(t)\\theta_i$, evaluated at $N$ discrete times through the basis matrix $A$ with $A_{ij}=p_i(t_j)$, so the network becomes $F(y,\\Theta A)$ in discretize-then-optimize and gradients are projected onto the basis by integrals in optimize-then-discretize. The mechanism that carries the argument is the conditioning of the Vandermonde matrix built from the basis: monomials give a Vandermonde matrix whose condition number grows fast with degree, while Legendre polynomials' orthogonality keeps the matrix well-conditioned. This conditioning difference is what the paper identifies as the reason monomial-parameterized training stalls and Legendre-parameterized training converges stably and cheaply.","core_discovery":"The paper's discovery is that replacing monomial basis functions with Legendre polynomials in the weight parameterization $\\theta_P(t)=\\sum_i p_i(t)\\theta_i$ changes the numerical conditioning of the training problem enough to determine whether convergence happens. Because the monomial Vandermonde matrix relating basis coefficients to time samples is ill-conditioned, small errors in evaluating the basis at time points become large weight errors, and training stalls; the orthogonal Legendre Vandermonde matrix stays well-conditioned, so gradients are projected cleanly onto the basis. In ResNets, third-order Legendre parameterization matches unparameterized accuracy while reducing the number of trainable weights, and it pairs especially well with Hamiltonian/Verlet architectures. In neural ODEs, where an adaptive integrator's function-evaluation count dominates runtime, Legendre parameterization cuts function evaluations by roughly an order of magnitude (about 8x on the DCR benchmark) while reaching similar or better loss. The authors' conclusion is that parameterization is a favorable expressivity-efficiency trade-off only when the basis is orthogonal.","pith_inferences":["A natural testable extension is to swap Legendre for Chebyshev or Fourier bases; if orthogonality is the real driver, similar efficiency gains should appear, and the paper's conditioning story would be confirmed.","Because all bases were compared under identical optimizer settings, an ablation that tunes learning rate and schedule separately for monomials would reveal whether monomials are intrinsically worse or just need different hyperparameters.","The conditioning mechanism predicts that as polynomial degree grows, monomial parameterization should degrade in stability faster than Legendre; a degree scan on a small synthetic ODE would quantify this directly.","In low-data surrogate regimes, parameterization acts as a form of regularization, so Legendre bases may also improve generalization as training set size shrinks; varying the training set size would test this."],"forward_implications":["A Legendre-parameterized ResNet can match the accuracy of an unparameterized ResNet with fewer trainable weights, reducing memory and training cost in memory-constrained settings.","In neural ODEs, switching from monomial to Legendre weight bases can reduce the number of adaptive-integrator function evaluations by nearly an order of magnitude, which dominates runtime.","Higher-degree Legendre bases (up to degree 6 in the reported experiments) increase expressivity and approach or beat unparameterized baselines, at the price of more parameters.","Both major training paradigms—discretize-then-optimize and optimize-then-discretize—benefit from the same orthogonal-basis parameterization, suggesting a unified default for continuous-time surrogates.","The Hamiltonian/Verlet architecture is a particularly good match for Legendre parameterization, since smooth orthogonal weights align with energy-preserving time integration."],"supporting_citations":[{"why":"Defines the neural ODE formulation and adjoint training that the paper generalizes to time-dependent weights.","marker":"[6]"},{"why":"Introduces polynomial-basis weight parameterization (monomial, Chebyshev, Legendre) for non-autonomous neural ODEs, the methodological starting point.","marker":"[13]"},{"why":"Supplies the stable ResNet architectures, including the Hamiltonian/Verlet continuous limit, used in the discretize-then-optimize experiments.","marker":"[10]"},{"why":"Provides the GNvpro variable-projection optimizer and the CDR and DCR benchmark datasets used for evaluation.","marker":"[16]"},{"why":"Models time-dependent weights with B-splines, establishing the smooth low-dimensional weight-control idea the paper transfers to polynomials.","marker":"[8]"},{"why":"Sets out the discretize-then-optimize versus optimize-then-discretize distinction the experiments are organized around.","marker":"[26]"},{"why":"Analyzes when neural ODE solutions are proper ODEs, supporting the static-versus-time-varying weight distinction.","marker":"[11]"},{"why":"Provides the checkpointed adjoint (ANODE) alternative that frames the memory and stability considerations in neural ODE training.","marker":"[28]"}],"fun_headline_variants":["Legendre weights fix ill-conditioning in neural ODEs","Orthogonal basis stabilizes continuous-time surrogate training","Legendre parameterization cuts neural ODE cost by 8x","Monomial basis fails; Legendre rescues neural dynamics"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the identical optimizer settings used for both bases are fair, so the monomial basis's poor convergence reflects the basis itself rather than a mismatch between monomials and those fixed hyperparameters.","fun_headline_variants_meta":{"raw":{"variants":["Legendre weights fix ill-conditioning in neural ODEs","Orthogonal basis stabilizes continuous-time surrogate training","Legendre parameterization cuts neural ODE cost by 8x","Monomial basis fails; Legendre rescues neural dynamics"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000361,"raw_usage":{"total_tokens":1939,"prompt_tokens":924,"completion_tokens":1015,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":946}},"tokens_in":540,"tokens_out":1015,"duration_ms":11521,"temperature":1.0,"reasoning_tokens":946,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T12:04:17.566607+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the DCR neural-ODE comparison with monomial parameterization using per-basis tuned learning rates, schedules, and a rescaled time interval, alongside a direct measurement of the monomial and Legendre Vandermonde condition numbers on the actual time grid; if monomial convergence and function-evaluation counts become comparable to Legendre's, the claimed conditioning advantage is not decisive.","supporting_citations":[{"cited_title":"Time Dependence in Non-Autonomous Neural ODEs","cited_arxiv_id":"2005.01906","evidence_quote":"Introduces polynomial-basis weight parameterization (monomial, Chebyshev, Legendre) for non-autonomous neural ODEs, the methodological starting point."},{"cited_title":"Stable Architectures for Deep Neural Networks","cited_arxiv_id":"1705.03341","evidence_quote":"Supplies the stable ResNet architectures, including the Hamiltonian/Verlet continuous limit, used in the discretize-then-optimize experiments."},{"cited_title":"Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection","cited_arxiv_id":"2007.13171","evidence_quote":"Provides the GNvpro variable-projection optimizer and the CDR and DCR benchmark datasets used for evaluation."},{"cited_title":"Spline parameterization of neural network controls for deep learning","cited_arxiv_id":"2103.00301","evidence_quote":"Models time-dependent weights with B-splines, establishing the smooth low-dimensional weight-control idea the paper transfers to polynomials."},{"cited_title":"ResNet After All? Neural ODEs and Their Numerical Solution","cited_arxiv_id":"2007.15386","evidence_quote":"Analyzes when neural ODE solutions are proper ODEs, supporting the static-versus-time-varying weight distinction."}],"review_version":1}