{"id":"cb74525a-4eab-410c-a72f-0c2b46fdd571","arxiv_id":"2508.11432","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Regularizing convolutional NODE weights to promote contractivity improves robustness to noise and adversarial attacks by up to 34 percentage points.","lead":"A new regularizer encourages convolutional neural ODEs to behave like contractive dynamical systems, so small input perturbations shrink instead of growing. On MNIST and FashionMNIST, the regularized models keep much higher accuracy under Gaussian noise and adversarial attacks than unregularized baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Trained CNODEs are never shown to satisfy the contractivity condition (8); the soft penalty (14) with γ=1 may not induce actual contractivity, leaving the claimed mechanism unverified.","rationale":"The reader identified the weakest assumption as the bridge between the continuous-time sufficient condition (8) and the discretized, soft-penalty training. I agree: this is the most load-bearing concern because the paper's theoretical justification for the regularizer is only a sufficient condition, and the regularizer is used as a soft penalty, not a constraint. The empirical results are convincing, but without post-training verification of (8), the causal claim that contractivity drives the robustness gains is unsupported. A generic L1 shrinkage baseline would further clarify whether the specific form of (14) matters. These concerns are addressable and do not invalidate the empirical findings, so the conditional verdict remains appropriate.","tokens_in":12957,"tokens_out":11481,"duration_ms":133745,"concrete_test":"After training, for each of the 10 seeds and each of the 10 Euler layers, compute V = max_i [ρ + 2κ W_ii + κ̄ Σ_{j≠i}(|W_ij| + |W_ji|)] using the actual convolution weights (either by forming the full W matrix or by using the filter-sum bounds from Lemma 1). If V > 0 for any layer, condition (8) fails. Report the fraction of layers/seeds with V < 0 for CNODE, vanilla NODE, and an L1-regularized baseline with matched clean accuracy. If CNODEs do not satisfy (8) but still show large robustness gains, the contractivity mechanism is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central mechanism is that contractivity of the NODE improves robustness, and the regularizer (14) is introduced to promote the sufficient condition (8). However, the experiments never verify that the trained CNODEs actually satisfy (8) or the equivalent Jacobian inequality (5). The regularizer is a soft penalty with γ=1; it does not guarantee (8) holds. If the learned convolution filters violate (8) for some layers or channels, then the observed robustness gains cannot be attributed to the contractivity guarantee of Theorem 1 - they might result from generic weight shrinkage induced by the absolute-value filter terms. Moreover, the pre/post-processing layers hα and gβ are not contractive, so even exact contractivity of (7) would not provide a global Lipschitz guarantee for the full classifier. The load-bearing but untested assumption is that the soft penalty sufficiently approximates condition (8) in the trained regime, and that this approximate contractivity is what drives the reported 34%/30% improvements.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes to improve the robustness of convolutional Neural ODEs (NODEs) by promoting contractivity through weight regularization. For the NODE class \\dot{x} = \\sigma(W_t x_t + b_t), Theorem 1 gives a Gershgorin-based sufficient condition (8) on W_t for contractivity. The authors then derive a filter-based regularizer (14) using Lemma 1, avoiding explicit Jacobian computation. They evaluate vanilla NODEs and regularized CNODEs on MNIST and FashionMNIST under Gaussian noise, salt-and-pepper noise, FGSM, and PGD attacks, reporting up to 34% and 30% accuracy improvements, with 10 seeds, hyperparameter sensitivity studies, and a transferability check.","tokens_in":13215,"tokens_out":8666,"duration_ms":88320,"significance":"If the theoretical bridge were sound, this would be a valuable, computationally cheap method for improving NODE robustness with a contraction-theoretic justification. The empirical study is extensive: multiple noise types and strengths, 10 seeds, ablations over \\rho, \\gamma, and convolution filter sizes, and a transferability study. Theorem 1 itself is correct given its assumptions. However, the connection between the proven sufficient condition and the implemented regularizer is flawed as written, and the trained networks are never verified to be contractive, so the central mechanism is not established by the current manuscript.","major_comments":[{"comment":"Condition (8) is equivalent to \\rho + 2\\kappa W_{ii} + \\bar\\kappa \\sum_{j\\ne i}(|W_{ij}|+|W_{ji}|) < 0. The regularizer in (11) uses \\phi = \\rho + 2(\\kappa+\\bar\\kappa)W_{ii} + \\bar\\kappa \\sum_{j=1}^n (|W_{ij}|+|W_{ji}|). If the sum includes j=i, the extra term 2\\bar\\kappa|W_{ii}| (when W_{ii}<0) makes \\phi smaller than the left-hand side of (8), so \\phi can be negative even when (8) is violated; if the sum is intended to run over j\\ne i, the coefficient of W_{ii} is wrong. Either way, the positive-part penalty is not a faithful proxy for the sufficient condition. The same issue propagates to (14). This needs to be corrected and the derivation stated explicitly.","section":"III-A, Eqs. (8) and (11)"},{"comment":"The paper never reports whether the trained CNODEs actually satisfy (8) or the Jacobian condition (5). Since \\gamma=1 is a soft penalty, the regularizer does not enforce contractivity. Without this verification, the observed robustness gains cannot be attributed to the contraction mechanism; they could be caused by generic weight shrinkage. I request a post-training evaluation of \\phi (or of the symmetric Jacobian's minimum eigenvalue) on the trained models, ideally per layer and across seeds.","section":"IV and Appendices V-B to V-E"},{"comment":"The contractivity bound (4) applies to the NODE state x_t as a function of the initial condition x_0. The full classifier is x_0 = h_\\alpha(z), y = g_\\beta(x_T). The paper explicitly omits h_\\alpha and g_\\beta, yet the attacks in Section IV are applied to the raw image z. Without Lipschitz bounds on h_\\alpha and g_\\beta, contractivity of the NODE alone does not yield an input-to-output robustness guarantee. This limitation should be stated and addressed, for example by reporting or constraining the Lipschitz constants of the pre- and post-processing layers.","section":"II-A"},{"comment":"Theorem 1 is for the continuous-time ODE (7). Training and inference use the Forward Euler discretization (2) with h=0.01. Contractivity of the continuous flow does not automatically transfer to the discretized map for a given step size. The paper should either prove a discrete-time counterpart or verify contractivity of the trained discrete network, for example by checking the Jacobian of one Euler step. This is especially important because all robustness experiments are performed on the discrete network.","section":"II-A, Eq. (2)"}],"minor_comments":[{"comment":"The notation \\lambda_{\\min}(\\Gamma(x)|_{x_i^k,k}) is unclear and not formally defined. Please clarify how the sampled states and time indices enter the Jacobian evaluation.","section":"III, Eq. (6)"},{"comment":"Omitting the factor P\\times H in (14) and absorbing it into \\gamma changes the effective regularization strength. For reproducibility, please report the effective product \\gamma/(P\\times H) or the actual value used.","section":"Remark 5 and IV"},{"comment":"The activation \\sigma(x)=0.1x+0.9\\log(1+e^x) is called 'smooth leaky ReLU'; the derivative range is stated correctly. Minor terminology: 'shorthand notion' should be 'shorthand notation', and the sentence following (14) is a fragment.","section":"Appendix V-A"},{"comment":"The table and accompanying text would be clearer if the rows were labeled 'white-box on CNODE' and 'transfer from vanilla NODE' in the caption itself, rather than only in the prose.","section":"Table III"}],"recommendation":"major_revision","confidential_remarks":"The main concern is not the experimental effort but the gap between the theory and the implemented regularizer. If the mismatch in (11) is a typo, it should be corrected; regardless, post-training verification of contractivity is essential to support the paper's central narrative. The self-citations to [26] are relevant and not excessive."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper delivers a useful, easy-to-compute regularizer for convolutional NODEs and shows big robustness gains on standard benchmarks. It also leaves the central mechanism unverified: we never learn whether the trained models actually satisfy the contractivity condition.\n\nWhat is new: the filter-level regularizer (14) is a concrete translation of a Gershgorin sufficient condition for contractivity (8) into a penalty on convolution filters, using Lemma 1 from NAIS-Net. That is a nice step: you avoid forming the full weight matrix W, so the penalty is cheap. The theorem behind it is straightforward and correct. The experiments are above the bar: ten seeds, Gaussian and salt-and-pepper noise, FGSM and PGD, a transferability study, and sensitivity analysis for rho and gamma. The gains are large (up to 34% accuracy on noisy images) and the standard deviations are smaller with the regularizer. The transfer result suggests the robustness is not from gradient masking.\n\nSoft spots, in order of severity. First, the paper never checks whether the trained CNODEs actually satisfy condition (8) or the Jacobian inequality (5). The regularizer is a soft penalty with gamma=1, so there is no guarantee it is active at the optimum. Without that check, the observed gains may come from generic weight shrinkage rather than from contractivity. The pre- and post-layers are also non-contractive, so even a contractive NODE would not give a global Lipschitz bound for the classifier. Second, the regularizer in (11) uses 2(kappa+bar_kappa) W_ii where the sufficient condition has 2 kappa W_ii; it is a more aggressive penalty, but the paper does not flag the discrepancy. Third, no code or data is released, which makes reproducibility harder. The hyperparameter selection uses test data for a grid check, though the insensitivity analysis mitigates that.\n\nNone of these are fatal. The empirical claim is credible, and the theory is a sound motivation even if the mechanism is not pinned down. The paper would benefit from a post-hoc check of contractivity of the trained weights, and from releasing the code. I would send it to peer review; the referee should focus on the mechanism gap rather than on the experiments, which are already solid.","headline":"Solid empirical paper with a clean new regularizer; the theory-mechanism link is unverified, but the results are strong enough to warrant peer review.","tokens_in":13649,"tokens_out":3557,"would_cite":true,"duration_ms":36797,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Contractivity-promoting regularization makes convolutional neural ODEs robust to noise and attacks.","keywords":["Neural ODE","contraction theory","adversarial robustness","regularization","convolutional NODE","Geršgorin theorem","MNIST","FashionMNIST"],"falsifier":"Check whether trained CNODEs actually satisfy condition (8) or the Jacobian inequality (5) on the sampled states. If a large fraction of trained models violate these conditions yet still show the reported robustness gains, the improvement is not explained by the proven contractivity condition.","tokens_in":12881,"feed_emoji":"🛡️","tokens_out":4691,"duration_ms":45197,"temperature":0.7,"pith_summary":"The paper argues that convolutional neural ODEs can be made substantially more robust to input noise and adversarial attacks by training them with a regularization term derived from contraction theory. Contractive dynamics guarantee that perturbed trajectories converge to each other exponentially fast, which limits how much a small input perturbation can change the output. The authors prove a sufficient condition on the weight matrices of NODEs with slope-restricted activations (Theorem 1) that guarantees contractivity, and translate it into a simple filter-based regularizer for convolutional layers. In experiments on MNIST and FashionMNIST, the regularized models improve average test accuracy by up to 34% under Gaussian noise and up to 30% under FGSM/PGD attacks, with no loss of clean accuracy.","feed_headline":"Neural ODEs get up to 34% more robust via contractivity penalty","feed_subtitle":"A contraction-theory weight penalty beats vanilla NODEs on noisy and adversarial images, with no drop in clean accuracy.","key_machinery":"The load-bearing object is the row-wise diagonal-dominance condition (8), $-\\rho - 2\\kappa W_{ii} - \\bar{\\kappa}\\sum_{j\\neq i}(|W_{ij}| + |W_{ji}|) > 0$. It is a sufficient condition, derived via the Ger\\v{s}gorin disk theorem, for the NODE (7) to be contractive. The work it does: it converts a global, state-dependent Jacobian inequality (5) into a simple constraint that depends only on the weights (or convolution filters), so contractivity can be promoted by adding the penalty (11) or its convolutional version (14) to the training loss.","core_discovery":"The central claim is that robustness of convolutional NODEs can be promoted by regularizing the network weights rather than computing and penalizing the Jacobian at every state. Theorem 1 states that if the activation slopes lie in $[\\kappa, \\bar{\\kappa}]$ and the weights satisfy $-\\rho - 2\\kappa W_{ii} - \\bar{\\kappa}\\sum_{j\\neq i}(|W_{ij}|+|W_{ji}|) > 0$ for every row $i$, then the NODE $\\dot{x} = \\sigma(W_t x + b)$ is contractive with rate $\\rho$. The proof uses the Ger\\v{s}gorin disk theorem on the Hermitian part of the Jacobian. Using Lemma 1, which relates the weight-matrix entries of a convolution to the convolution filters, the condition becomes a per-filter penalty $\\psi(C^i_j, \\rho,","pith_inferences":["Inference: Because condition (8) is sufficient but not necessary, the robustness gains in the experiments may be obtainable with even weaker penalties, or conversely, the training-time $\\rho$ may be conservative compared with the actual contraction of trained networks.","Inference: The same filter-based penalty could plausibly extend to other continuous-depth architectures (e.g., Hamiltonian NODEs or implicit networks) whose dynamics can be written with slope-restricted activations, though the paper only treats the NODE form (7).","Inference: The transferability result suggests contractivity regularization does not rely on gradient masking, so it may compose with adversarial training; the paper does not test this combination, but it is a natural next experiment.","Inference: A direct verification of condition (8) or the Jacobian inequality (5) on trained CNODE weights would close the gap between the continuous-time theorem and the discrete-time model actually used; the paper does not report such measurements."],"forward_implications":["If true, robustness for NODE-based classifiers can be bought with a cheap weight penalty, without adversarial training or Jacobian computations.","Contractive NODEs remain accurate on clean data while reducing sensitivity to perturbation strength, as the reported standard deviations are the same or lower than vanilla NODEs in almost all experiments.","The sufficient condition can be used to parameterize contractive NODEs by construction (Remark 3), not just as a soft penalty.","The regularizer generalizes across convolution filter sizes (3×3, 5×5, 7×7) and is insensitive to the contraction rate $\\rho$ and penalty weight $\\gamma$ per the appendix studies."],"supporting_citations":[{"why":"Supplies the contraction-theory machinery and the iff condition (5) between contractivity and the Hermitian part of the Jacobian, the basis of all proposed regularizers.","marker":"[16]"},{"why":"Provides the Ger\\v{s}gorin disk theorem used to derive the row-wise sufficient condition (8) in Theorem 1.","marker":"[34]"},{"why":"Provides Lemma 1, which relates the weight-matrix entries of a convolution to sums of filter coefficients, enabling the filter-based regularizer (14).","marker":"[36]"},{"why":"Introduces Neural ODEs and the dynamical-system formulation (1) that the paper robustifies.","marker":"[10]"},{"why":"Prior contractive Hamiltonian NODE work by the same group, serving as the direct baseline and motivation for promoting contractivity by design.","marker":"[26]"},{"why":"Defines the FGSM attack used to generate one class of adversarial test perturbations.","marker":"[4]"},{"why":"Defines the PGD attack used to generate the other class of adversarial test perturbations.","marker":"[38]"},{"why":"Raises the obfuscated-gradients concern that motivates the transferability study showing CNODE robustness is not due to gradient masking.","marker":"[39]"},{"why":"Earlier study of NODE robustness against random perturbations, providing the context and baseline that this work extends with contraction theory.","marker":"[18]"}],"fun_headline_variants":["Weight penalty makes neural ODEs 34% more robust","Contractivity regularization boosts NODE robustness","Slope-restricted NODEs get robust via weight regularity","Contraction-based penalty strengthens ODE nets against noise","NODE robustness up 34% with contractivity weight penalty"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The proof guarantees contractivity for the continuous-time ODE under condition (8), but training uses a soft penalty on discretized weights; the paper assumes this penalty makes the trained networks actually contractive enough, and that the non-contractive encoder and decoder layers do not undo the robustness.","fun_headline_variants_meta":{"raw":{"variants":["Weight penalty makes neural ODEs 34% more robust","Contractivity regularization boosts NODE robustness","Slope-restricted NODEs get robust via weight regularity","Contraction-based penalty strengthens ODE nets against noise","NODE robustness up 34% with contractivity weight penalty"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000169,"raw_usage":{"total_tokens":1103,"prompt_tokens":750,"completion_tokens":353,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":274}},"tokens_in":494,"tokens_out":353,"duration_ms":4484,"temperature":1.0,"reasoning_tokens":274,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T19:55:59.163949+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Check whether trained CNODEs actually satisfy condition (8) or the Jacobian inequality (5) on the sampled states. If a large fraction of trained models violate these conditions yet still show the reported robustness gains, the improvement is not explained by the proven contractivity condition.","supporting_citations":[{"cited_title":"Contraction theory for nonlinear stability analysis and learning-based control: A tutorial overview,","cited_arxiv_id":null,"evidence_quote":"Supplies the contraction-theory machinery and the iff condition (5) between contractivity and the Hermitian part of the Jacobian, the basis of all proposed regularizers."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Ger\\v{s}gorin disk theorem used to derive the row-wise sufficient condition (8) in Theorem 1."},{"cited_title":"Nais-net: Stable deep networks from non-autonomous differential equations,","cited_arxiv_id":null,"evidence_quote":"Provides Lemma 1, which relates the weight-matrix entries of a convolution to sums of filter coefficients, enabling the filter-based regularizer (14)."},{"cited_title":"Neural ordinary differential equations,","cited_arxiv_id":null,"evidence_quote":"Introduces Neural ODEs and the dynamical-system formulation (1) that the paper robustifies."},{"cited_title":"Robust classification using contractive Hamiltonian neural ODEs,","cited_arxiv_id":null,"evidence_quote":"Prior contractive Hamiltonian NODE work by the same group, serving as the direct baseline and motivation for promoting contractivity by design."},{"cited_title":"Adversarial robustness of stabilized neural ode might be from obfuscated gradients,","cited_arxiv_id":null,"evidence_quote":"Raises the obfuscated-gradients concern that motivates the transferability study showing CNODE robustness is not due to gradient masking."},{"cited_title":"On robustness of neural ordinary differential equations,","cited_arxiv_id":null,"evidence_quote":"Earlier study of NODE robustness against random perturbations, providing the context and baseline that this work extends with contraction theory."}],"review_version":1}