REVIEW 3 major objections 5 minor 41 references
Adversarial Robustness of Stabilized NeuralODEs Might be from Obfuscated Gradients
T0 review · 3 major / 5 minor · reviewed 2026-08-27 · deepseek-v4-flash
Pith's one-line read The paper argues that the apparent adversarial robustness of stabilized neural ODE networks is an artifact of gradient obfuscation in adaptive-stepsize solvers, not genuine robustness.
desk verdict A useful negative result with one overbroad headline: the ablation cleanly shows adaptive-step solvers cause gradient masking, but CW/SPSA are only tested at coarse tolerance, so 'mainly from' overstates the evidence. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the stabilized ODE block $\frac{d}{dt}[x;z] = \sigma\big(A_{k+1}[x;z]\big)$ with skew-symmetric $A_{k+1}$ plus damping $\gamma I$; the stability proof works with the Jacobian $J = D(A_{k+1}-\gamma I)$ and a similarity transform to $D^{1/2}(A_{k+1}-\gamma I)D^{1/2}$, where $D$ is the positive diagonal matrix of activation derivatives. The mechanism that produces the robustness numbers is the adaptive-stepsize solver (DOPRI5, Bosh3, adaptive Heun); its error-tolerance-driven step selection perturbs the gradient of the loss and masks it from PGD. Fixed-step solvers remove the masking and the robustness vanishes.
What would settle it
Train the same SOBlock or SONet with a fixed-step solver (Euler or RK4) instead of an adaptive one and run PGD-20; if robust accuracy stays well above zero, the claim that adaptive-stepsize solvers are the source of robustness is refuted. The paper's own experiment finds it drops to 0%.
Extended reading notes
Core claim
The paper proposes a Stabilized Neural ODE Network (SONet), in which each ODE block uses two weight matrices constrained by skew-symmetry ($W^{(2)} = -W^{(1)\top}$) and a small damping term $\gamma$, so the linearized system matrix is $A_{k+1} = \begin{bmatrix}0 & -W^\top \\ W & 0\end{bmatrix} - \gamma I$. Theorem 3.1 claims these blocks are input-output stable in the Lyapunov sense provided the activation is strictly increasing and the damping is positive. Under natural training, the architecture and a variant replacing only the first ResNet layer (SOBlock) attain PGD-20 $\ell_\infty$ robust accuracy around 62% on CIFAR-10, higher than ResNet10-TRADES's 45% at the same epsilon, while keeping natural accuracy near 91%. The paper then shows this robustness disappears under CW-infinity attacks (0% for SOBlock) and SPSA attacks (around 12–15%), and that replacing the adaptive solver with fixed-step Euler or RK4 collapses PGD robustness to 0%. It concludes that adaptive step-size selection in numerical integration obfuscates the loss gradient, so the PGD robustness numbers are not trustworthy.
Load-bearing premise
The stability proof assumes that a positive diagonal similarity transform preserves the condition that all eigenvalues have negative real parts, which does not hold for general non-normal matrices; if this step is wrong, the Lyapunov guarantee does not follow from the stated argument.
Editorial extensions
If this is right
- PGD-based robustness claims for ODE networks trained with adaptive solvers should be re-checked with CW or SPSA attacks before being trusted.
- Natural training of a stable ODE block can outperform TRADES on PGD metrics, but the paper indicates that outperformance is an artifact of solver-induced gradient masking, not real robustness.
- The Lyapunov stability of the ODE architecture does not by itself confer adversarial robustness under white-box attacks.
- Replacing just the first layer of a ResNet with a stabilized ODE block can give high PGD numbers, so architectural changes can mask vulnerabilities rather than remove them.
- Robustness evaluation of ODE-based defenses should include gradient-free attacks; otherwise the defense may be substantially overestimated.
Reading between the lines
- The same gradient-masking effect likely applies to other neural ODE models using adaptive-stepsize solvers, so previously reported PGD robustness for such models may need revisiting.
- A genuinely robust ODE defense would need to combine a stable architecture with adversarial training using attacks that survive gradient obfuscation, or to explicitly regularize the solver's sensitivity to input perturbations.
- The claimed Lyapunov stability can likely be proved by a direct Lyapunov-function argument using the fact that the symmetric part of $A_{k+1}-\gamma I$ is $-\gamma I$, avoiding the questionable eigenvalue step.
- One testable extension is to measure input-loss gradient norms across solver tolerances: if larger tolerances systematically produce noisier or smaller gradients, that would confirm the masking mechanism.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes Stabilized ODE Networks (SONet), whose ODE blocks use skew-symmetric weight pairs and a damping parameter gamma, and claims Lyapunov stability for this architecture. It reports that naturally trained SONet and SOBlock achieve nontrivial PGD robustness on CIFAR-10, in some settings exceeding TRADES-trained ResNets. The central explanatory claim is that this apparent robustness is largely an artifact of gradient masking caused by adaptive-stepsize numerical ODE solvers: all adaptive-step solvers tested yield PGD robustness, fixed-step Euler/RK4 yield 0%, and CW-infinity and SPSA attacks break the adaptive-solver models at the default tolerance. The paper therefore concludes that the adversarial robustness of ODE-based networks mainly comes from obfuscated gradients in adaptive-step integration.
Significance. If established, the central negative result is important: it identifies a solver-induced gradient-masking mechanism in Neural ODEs and cautions against interpreting PGD robustness of adaptive-step ODE networks as genuine robustness. The paper's ablation design is clean: the fixed-step versus adaptive-step comparison controls the architecture and isolates the solver, and the CW/SPSA experiments provide independent falsification tests. The public source-code links are a further strength. However, the current evidence for the 'mainly comes from obfuscated gradients' claim is incomplete at tight solver tolerances, and the stability proof in Section 3 has a genuine gap. The significance is real but the strength of the conclusion currently exceeds the evidence.
major comments (3)
- [Section 3, proof of Theorem 3.1, Eq. (5)] The proof of Theorem 3.1 contains an invalid eigenvalue inference. From Re lambda_i(A_{k+1} - gamma I) < 0 and D_{k+1} positive diagonal, the manuscript concludes Re lambda_i(D^{1/2}(A_{k+1} - gamma I)D^{1/2}) < 0. This does not follow for non-normal matrices; eigenvalues are not preserved under diagonal congruence. In this particular construction the claim can be rescued by observing that D^{1/2} A D^{1/2} is skew-symmetric and -gamma D is negative definite, so the field of values lies in the open left half-plane, but the written derivation omits that argument. In addition, the subsequent conclusion ||(x(t),z(t))|| <= ||(x(0),z(0))|| for the nonlinear system does not follow from the eigenvalue condition on the linearization alone; a Lyapunov-function or contraction argument is needed. Since Theorem 3.1 is a stated main contribution, the proof must be rewritten.
- [Section 5.1 versus Section 3] All reported experiments set gamma = 0 (Section 5.1: 'we set the constant gamma in (3) to be 0'), while Theorem 3.1 assumes a positive damping factor ('positive damping factor gamma is small'). The evaluated architecture is therefore not the provably stable system analyzed in the theorem. Please either provide experiments with gamma > 0 for the stability claims or explicitly separate the stability theorem from the experimental configuration.
- [Section 6.2, Tables 2-4] The CW-infinity and SPSA results are reported only for DOPRI5 with tol = 0.1. Table 2 shows that SOBlock at tol = 0.001 still retains 63.87% PGD20 and 46.20% PGD1000 robust accuracy, and Table 3 shows that the adaptive-step-size oscillations are about an order of magnitude smaller at tol = 0.001. Without CW-infinity and SPSA results at tighter tolerances, the abstract's claim that robustness 'mainly comes from' obfuscated gradients is an overgeneralization. Please add CW-infinity and SPSA attacks at tol = 0.01 and 0.001 for both SOBlock and ODENet, and report whether robust accuracy collapses at all tolerances.
minor comments (5)
- [Abstract and Section 7] The title and some passages say 'might be' from obfuscated gradients, while the Abstract and Section 7 say the robustness 'mainly comes from' or 'is mainly due to' obfuscated gradients; the wording should be aligned with the evidence actually presented.
- [Tables 1, 2, and 4] No standard deviations or number of training seeds are reported; please add error bars over seeds and attack restarts, or explicitly state that single runs are reported.
- [Table 2] The fixed-step solvers are only tested at h = 1; a step-size sweep for Euler and RK4 would make the claim that robustness is associated only with adaptive step sizes stronger.
- [Section 6.1.2] The discussion of over-enlarging error tolerance is qualitative; specify the tolerance at which natural accuracy and robust accuracy begin to degrade for each solver.
- [Abstract and Section 5.1] There are typos and incomplete phrases, including 'natrual' in Section 5.1 and 'the possible mechanism underlying such .' in the Abstract; a careful proofread is needed.
Circularity Check
No significant circularity: the gradient-masking claim is tested against external attacks and solver ablations, not fitted or assumed.
full rationale
The paper's central claim is that adaptive-stepsize ODE solvers create obfuscated gradients and that this explains the apparent PGD robustness of ODE-based networks. This claim is not derived from its own inputs: it is supported by ablation experiments across solvers (Euler, RK4, adaptive Heun, Bosh3, DOPRI5) and by evaluation under CW-infinity and SPSA attacks, which are independent of the PGD gradients that the paper argues are masked. The stability theorem for SONet is an architectural motivation, but the robustness attribution does not assume the theorem's conclusion; the later experiments would apply even if the theorem were weaker. The only self-citation is TRADES, used as a comparison baseline and as an external, code-available benchmark; it is not used to justify the gradient-masking explanation. A mathematical gap in the proof of Theorem 3.1, where the eigenvalue comparison in Eqn. (5) is not justified for non-normal matrices, is a correctness concern rather than a circularity, because no conclusion is imported from that step into the later empirical claim. No fitted parameter is renamed as a prediction, and no known result is repackaged as new. Accordingly, no circular step meeting the quoted-evidence standard was found.
Assumptions & free parameters
free parameters (2)
- gamma (damping factor) =
0
- DOPRI5 error tolerance tol =
0.1
assumptions (4)
- domain assumption Activation function is strictly monotonically increasing (sigma' > 0).
- standard math The ODE block with W^{(2)} = -W^{(1)T} and gamma >= 0 is stable if the symmetric part of the Jacobian is negative definite.
- domain assumption The adaptive-step solver's internal step-size selection is differentiable enough for backpropagation to compute a loss gradient that PGD senses.
- domain assumption Ablation across solvers isolates the adaptive-step mechanism from solver order or accuracy.
Cite this review
Pith. "Pith review of Adversarial Robustness of Stabilized NeuralODEs Might be from Obfuscated Gradients." pith.science (2026). https://pith.science/paper/IGWSN7IC
@misc{pith2026200913145,
author = {Pith},
title = {Pith review of: Adversarial Robustness of Stabilized NeuralODEs Might be from Obfuscated Gradients},
year = {2026},
howpublished = {\url{https://pith.science/paper/IGWSN7IC}},
note = {Machine review of arXiv:2009.13145}
}
abstract
In this paper we introduce a provably stable architecture for Neural Ordinary Differential Equations (ODEs) which achieves non-trivial adversarial robustness under white-box adversarial attacks even when the network is trained naturally. For most existing defense methods withstanding strong white-box attacks, to improve robustness of neural networks, they need to be trained adversarially, hence have to strike a trade-off between natural accuracy and adversarial robustness. Inspired by dynamical system theory, we design a stabilized neural ODE network named SONet whose ODE blocks are skew-symmetric and proved to be input-output stable. With natural training, SONet can achieve comparable robustness with the state-of-the-art adversarial defense methods, without sacrificing natural accuracy. Even replacing only the first layer of a ResNet by such a ODE block can exhibit further improvement in robustness, e.g., under PGD-20 ($\ell_\infty=0.031$) attack on CIFAR-10 dataset, it achieves 91.57\% and natural accuracy and 62.35\% robust accuracy, while a counterpart architecture of ResNet trained with TRADES achieves natural and robust accuracy 76.29\% and 45.24\%, respectively. To understand possible reasons behind this surprisingly good result, we further explore the possible mechanism underlying such an adversarial robustness. We show that the adaptive stepsize numerical ODE solver, DOPRI5, has a gradient masking effect that fails the PGD attacks which are sensitive to gradient information of training loss; on the other hand, it cannot fool the CW attack of robust gradients and the SPSA attack that is gradient-free. This provides a new explanation that the adversarial robustness of ODE-based networks mainly comes from the obfuscated gradients in numerical ODE solvers.
Figures
Reference graph
Works this paper leans on
-
[1]
Feedback systems: an introduction for scientists and engineers
Karl Johan Astr \"o m and Richard M Murray. Feedback systems: an introduction for scientists and engineers . Princeton university press, 2010
work page 2010
-
[2]
Neural ODEs as the Deep Limit of ResNets with constant weights
Benny Avelin and Kaj Nystr \"o m. Neural ODEs as the deep limit of ResNets with constant weights. arXiv preprint arXiv:1906.12183 , 2019
work page Pith review arXiv 1906
-
[3]
Jean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In Advances in Neural Information Processing Systems , pages 12192--12202, 2019
work page 2019
-
[4]
Wieland Brendel, Jonas Rauber, Alexey Kurakin, Nicolas Papernot, Behar Veliqi, Sharada P Mohanty, Florian Laurent, Marcel Salath \'e , Matthias Bethge, Yaodong Yu, et al. Adversarial vision challenge. In The NeurIPS'18 Competition , pages 129--153. Springer, 2020
work page 2020
-
[5]
Provable robustness of relu networks via maximization of linear regions
Francesco Croce, Maksym Andriushchenko, and Matthias Hein. Provable robustness of relu networks via maximization of linear regions. In International Conference on Artificial Intelligence and Statistics , 2018
work page 2018
-
[6]
On Evaluating Adversarial Robustness
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705 , 2019
work page Pith review arXiv 1902
-
[7]
Parseval networks: Improving robustness to adversarial examples
Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. Parseval networks: Improving robustness to adversarial examples. In International Conference on Machine Learning , 2017
work page 2017
-
[8]
M. Frank Callier and A. Charles Desoer. Linear System Theory . Springer-Verlag, 1991
work page 1991
Show all 41 references
-
[9]
Neural ordinary differential equations
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems , pages 6571--6583, 2018
2018
-
[10]
Certified adversarial robustness via randomized smoothing
Jeremy M Cohen, Elan Rosenfeld, and J Zico Kolter. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning , 2019
2019
-
[11]
Unlabeled data improves adversarial robustness
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang. Unlabeled data improves adversarial robustness. In Advances in Neural Information Processing Systems , pages 11190--11201, 2019
2019
-
[12]
Towards evaluating the robustness of neural networks
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy , pages 39--57, 2017
2017
-
[13]
J. R. Dormand and P. J. Prince. A family of embedded runge-kutta formulae. Journal of Computational and Applied Mathematics , 6(1):19–--26, 1980
1980
-
[14]
Robust physical-world attacks on deep learning visual classification
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-world attacks on deep learning visual classification. In IEEE Conference on Computer Vision and Pattern Recognition , pages 1625--1...
2018
-
[15]
Stable architectures for deep neural networks
Eldad Haber and Lars Ruthotto. Stable architectures for deep neural networks. Inverse Problems , 34(1):014004, 2017
2017
-
[16]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE conference on computer vision and pattern recognition , pages 770--778, 2016
2016
-
[17]
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning , pages 448--456, 2015
2015
-
[18]
Adversarial examples for evaluating reading comprehension systems
Robin Jia and Percy Liang. Adversarial examples for evaluating reading comprehension systems. In Empirical Methods in Natural Language Processing , 2017
2017
-
[19]
Provable defenses against adversarial examples via the convex outer adversarial polytope
J Zico Kolter and Eric Wong. Provable defenses against adversarial examples via the convex outer adversarial polytope. In International Conference on Machine Learning , 2018
2018
-
[20]
On norm-agnostic robustness of adversarial training
Bai Li, Changyou Chen, Wenlin Wang, and Lawrence Carin. On norm-agnostic robustness of adversarial training. arXiv preprint arXiv:1905.06455 , 2019
1905 arXiv
-
[21]
Implicit euler skip connections: Enhancing adversarial robustness via numerical stability
Mingjie Li, Lingshen He, and Zhouchen Lin. Implicit euler skip connections: Enhancing adversarial robustness via numerical stability. In International Conference on Machine Learning , 2020
2020
-
[22]
MagNet : a two-pronged defense against adversarial examples
Dongyu Meng and Hao Chen. MagNet : a two-pronged defense against adversarial examples. In ACM SIGSAC Conference on Computer and Communications Security , pages 135--147, 2017
2017
-
[23]
Towards deep learning models resistant to adversarial attacks
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations , 2018
2018
-
[24]
Haifeng Qian and Mark N. Wegman. l_2 -nonexpansive neural networks. In International Conference on Learning Representations , 2019
2019
-
[25]
Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition
Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K Reiter. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In ACM SIGSAC Conference on Computer and Communications Security , pages 1528--1540, 2016
2016
-
[26]
Pixeldefend: Leveraging generative models to understand and defend against adversarial examples
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. In International Conference on Learning Representations , 2018
2018
-
[27]
Peernets: Exploiting peer wisdom against adversarial attacks
Jan Svoboda, Jonathan Masci, Federico Monti, Michael Bronstein, and Leonidas Guibas. Peernets: Exploiting peer wisdom against adversarial attacks. In International Conference on Learning Representations , 2019
2019
-
[28]
Adversarially robust generalization requires more data
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander M a dry. Adversarially robust generalization requires more data. In Advances in Neural Information Processing Systems 31 , pages 5019--5031, 2018
2018
-
[29]
Robustness may be at odds with accuracy
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. In International Conference on Learning Representations , 2019
2019
-
[30]
Deep limits of residual neural networks
Matthew Thorpe and Yves van Gennip. Deep limits of residual neural networks. arXiv preprint arXiv:1810.11741 , 2018
2018 arXiv
-
[31]
Fooling automated surveillance cameras: adversarial patches to attack person detection
Simen Thys, Wiebe Van Ranst, and Toon Goedem \'e . Fooling automated surveillance cameras: adversarial patches to attack person detection. In IEEE Conference on Computer Vision and Pattern Recognition Workshops , 2019
2019
-
[32]
Adversarial risk and the dangers of evaluating against weak attacks
Jonathan Uesato, Brendan O'Donoghue, Aaron van den Oord, and Pushmeet Kohli. Adversarial risk and the dangers of evaluating against weak attacks. arXiv preprint arXiv:1802.05666 , 2018
2018 arXiv
-
[33]
Bao Wang, Binjie Yuan, Zuoqiang Shi, and Stanley J. Osher. Resnets ensemble via the feynman-kac formalism to improve natural and robust accuracies. In Advances in Neural Information Processing Systems 32 (NIPS 2019) , 2019
2019
-
[34]
Feature denoising for improving adversarial robustness
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan L Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 501--509, 2019
2019
-
[35]
Adversarial examples for semantic segmentation and object detection
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille. Adversarial examples for semantic segmentation and object detection. In International Conference on Computer Vision , 2017
2017
-
[36]
On robustness of neural ordinary differential equations
Hanshu Yan, Jiawei Du, Vincent YF Tan, and Jiashi Feng. On robustness of neural ordinary differential equations. arXiv preprint arXiv:1910.05513 , 2019
1910 arXiv
-
[37]
Adversarially robust generalization just requires more unlabeled data
Runtian Zhai, Tianle Cai, Di He, Chen Dan, Kun He, John Hopcroft, and Liwei Wang. Adversarially robust generalization just requires more unlabeled data. arXiv preprint arXiv:1906.00555 , 2019
1906 arXiv
-
[38]
Improving the robustness of deep neural networks via stability training
Stephan Zheng, Yang Song, Thomas Leung, and Ian Goodfellow. Improving the robustness of deep neural networks via stability training. In IEEE Conference on Computer Vision and Pattern Recognition , pages 4480--4488, 2016
2016
-
[39]
Deep neural networks with multi-branch architectures are intrinsically less non-convex
Hongyang Zhang, Junru Shao, and Ruslan Salakhutdinov. Deep neural networks with multi-branch architectures are intrinsically less non-convex. In International Conference on Artificial Intelligence and Statistics , pages 1099--1109, 2019
2019
-
[40]
Theoretically principled trade-off between robustness and accuracy
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael I Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning , 2019
2019
-
[41]
You only propagate once: Painless adversarial training using maximal principle
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Painless adversarial training using maximal principle. In Advances in Neural Information Processing Systems 32 (NeurIPS) , 2019
2019
Reviewed August 27, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.