REVIEW 4 major objections 6 minor 32 references
A Data-Driven Method for Modeling Creep-Fatigue Stress-Strain Behavior Using Neural ODEs
T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read A neural ODE trained directly on 19 creep-fatigue experiments reproduces Alloy 617's measured stress-strain behavior more accurately than the standard Chaboche model, and a symbolic-regression version recovers an interpretable polynomial…
desk verdict A useful application of Neural ODEs to real creep-fatigue data, but the per-curve normalization leaks validation information and the Chaboche/SINDy outputs are missing, so the headline accuracy claim isn't yet supported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Neural ODE used as a constitutive law: instead of postulating an inelastic flow rule and hardening equations, one takes the time derivative of the state $\boldsymbol{x}=(\sigma,h)$ to be a neural network $f(\boldsymbol{x},\theta,u)$ and integrates it forward in time. In the black-box model the network maps $(\sigma,h,\varepsilon)$ to $\dot\sigma$; in the neural flow rule model the network supplies the inelastic strain-rate-like history term $\dot h$ inside Hooke's law. Training uses an implicit, adjoint-based solver so that the stiff material ODEs can be fit on GPUs, and robust initialization is obtained by repeated He initialization with Monte Carlo selection of the lowest-loss start. Interpretability is recovered with SINDy, which takes trajectories of the trained network and sparse-regresses them against a third-order polynomial library to produce an explicit polynomial ODE system $\dot{\boldsymbol{x}}=\boldsymbol{\Theta}(\boldsymbol{x})\boldsymbol{\Phi}$.
What would settle it
Retrain the black-box and Chaboche models with a fixed normalization, for example a global strain range and a stress range capped from the training set, hold out the same four curves, and compare validation MSE; if the neural ODE no longer beats Chaboche, the reported superiority is an artifact of per-curve extrema leakage. A second test would be to run one new creep-fatigue test outside the training grid and check whether either model's prediction error grows faster than Chaboche's.
Extended reading notes
Core claim
On its own terms, the paper's central discovery is that a Neural ODE can act as a material model: the stress rate (and, in the black-box variant, the stress rate plus auxiliary internal variables) is written as a learned function of current stress, an internal state, and the applied strain, and the system is integrated with an implicit solver. In the black-box model the right-hand side is a residual network plus a learned linear driving term $P\varepsilon$, so the network must represent the entire constitutive response. In the neural flow rule model the equation is $\dot\sigma = E(\dot\varepsilon - \dot h)$ with $\dot h$ learned by a residual network, embedding the elastic/plastic decomposition of classical flow rules. Against 19 experiments the black-box model attains the lowest mean-squared error of all models, the flow-rule model is comparable to or better than the standard Chaboche model, and the polynomial model recovered by SINDy reaches roughly Chaboche-level accuracy. The paper presents this as evidence that data-driven ODE models can replace the need to specify a material-specific constitutive form.
Load-bearing premise
The load-bearing premise is that each stress-strain curve is normalized by its own measured global minimum and maximum before training and validation, so the model is effectively told the full range of every test in advance; in a real deployment the future extrema would not be known, and the claimed accuracy may not transfer.
Editorial extensions
If this is right
- If the claims hold, a black-box Neural ODE can serve as a drop-in constitutive model for strain-controlled cyclic loading, matching or beating the standard Chaboche model without any assumed hardening or flow-rule form.
- Because it captures the first-cycle overstress, the black-box model describes transient cyclic behavior that the standard Chaboche form structurally cannot represent.
- The SINDy-recovered polynomial ODE gives a compact, transparent equation that could be tabulated or inserted into a finite-element material routine with accuracy close to the empirical baseline.
- The neural flow rule model, by preserving Hooke's law, offers a middle path: less flexible than the black box but more robust with limited data and closer to classical mechanical structure.
- The framework, if it generalizes, would reduce the cost of developing constitutive models for new or less-studied materials, since only uniaxial experimental data and no material-specific derivations are needed.
Reading between the lines
- My inference: the headline comparison rests on per-curve min/max normalization, so the reported MSE advantage is for reconstructing curves whose range is already known; a fairer deployment test would normalize with a fixed range learned from training tests only, and the ranking could change.
- My inference: the four validation curves are held out from parameter fitting but not from normalization, so true extrapolation to unseen strain ranges, hold times, or temperatures is not established by the paper, which acknowledges that out-of-sample testing is not feasible.
- My inference: the explicit SINDy polynomial could be inspected term by term to see whether any discovered terms resemble known backstress or recovery terms; if so, the pipeline becomes a hypothesis generator for mechanistic constitutive laws rather than only a predictor.
- My inference: the proposed 3D extension via J2 flow theory is assumed rather than validated; confirming it would require multiaxial or component-level experiments, since uniaxial data cannot identify the tensor flow direction.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes two Neural ODE architectures, a fully black-box model and a hybrid 'Neural flow rule' model, for uniaxial creep-fatigue stress-strain response of Alloy 617 at 950°C. The models are trained on 15 experimental stress-strain curves and evaluated on 4, and compared against a calibrated three-backstress Chaboche model. The authors report that the black-box Neural ODE exceeds Chaboche accuracy, that the Neural flow rule model is comparable or better than Chaboche, and that a SINDy-derived polynomial model extracted from the black-box network achieves accuracy comparable to Chaboche while being interpretable. The paper also presents a state-space stability analysis of the SINDy model.
Significance. If the central claims are valid, the paper would make a useful contribution: it applies Neural ODEs directly to experimental creep-fatigue data, uses a small dataset (a realistic constraint), provides an open-source implicit ODE solver (pyoptmat), and attempts to interpret the learned dynamics via SINDy. The explicit modeling of the elastic part in the Neural flow rule model is a sensible hybrid design. However, the evaluation methodology currently does not support the abstract's claim of 'precisely capture' and 'exceeding the standard Chaboche model's accuracy.' The per-curve normalization leaks future information into validation, the Chaboche baseline parameters are not actually reported, and the SINDy model is fit to the network's own trajectories rather than to independent experimental data. These issues are load-bearing for the paper's main conclusions.
major comments (4)
- [Section 3.1, Eq. (12)] The normalization described in Eq. (12) uses each stress-strain curve's own global minimum and maximum before the training/validation split. For the stress variable, these extrema are measured outputs of the very validation curve being predicted, so the validation procedure assumes knowledge of future stress values that would not be available at deployment. The MSE comparison in Fig. 14 and the abstract's superiority claim are therefore not a genuine out-of-sample evaluation. The authors should redo the evaluation with normalization constants computed only from training data (or, better, with leave-one-out cross-validation where normalization is refit on each training fold) and report results in physical units. The current Conclusion statement that 'out-of-sample testing is not feasible' is in tension with the paper's earlier validation claims.
- [Section 4, Table 2] Table 2 is captioned 'Calibrated Parameters for Chaboche Model' but its content is a list of experimental conditions, not the calibrated Chaboche parameters (E, sigma0, K, eta, n, C_i, gamma_i, delta, R). Without these numbers, the Chaboche baseline is not reproducible, and the reader cannot judge whether the calibration is of comparable quality to the neural models or whether the comparison is fair. The authors should either report the calibrated parameters or clearly indicate where they can be found, and should also state which of the 19 experiments were used for calibration (e.g., all 19, or only the 15 training curves, and with what loss).
- [Section 5.1] The SINDy model is fit to the black-box neural network's own trajectories (X and Xdot extracted from the trained network), not directly to the experimental data. As a result, the accuracy comparison of the 'interpretable model' against experimental data in Fig. 16 is not an independent test of the discovered equations; it measures how well a cubic polynomial approximates the network, compounded by the network's own fit quality and the normalization leakage. The claim that a 'purely data-driven approach achieved accuracy close to the Chaboche model' is therefore overstated. Additionally, the detailed polynomial equations promised after Eq. (18) are not actually printed, so the interpretability and stability analysis (Eqs. (19)-(20)) cannot be verified by the reader. The authors should report the discovered equations and, ideally, re-fit or at least evaluate the SINDy model directly on experimental data.
- [Section 2.2 / 4.2, Validation methodology] The evaluation rests on a single 15/4 split with no cross-validation, no repeated splits, and no error bars, as shown in Fig. 14. With only four validation curves and a per-curve normalization that leaks information, the claimed 'superior predictive capability' of the black-box model is not statistically supported. The authors should provide either a leave-one-out evaluation, repeated random splits with reported means and standard deviations, or at minimum a clear statement that the 'validation' here is not predictive but an assessment of interpolation within the same experimental database. The Section 6 statement that 'out-of-sample testing is not feasible for this paper' should be reflected in a more guarded version of the abstract's claim.
minor comments (6)
- [Section 2.2, Eq. (9)] The internal state h is introduced as 'arbitrary' and its dimension is never specified. Since h evolves through a neural network, its initialization and the identifiability of the combined (sigma, h) system should be discussed; otherwise the SINDy extraction and the physical meaning of the derived internal variables x1 and x2 remain ambiguous.
- [Section 2.4.2] The Monte Carlo initialization procedure selects the initial point with the lowest training loss among 30 He-initialized candidates. This selection is made on the same training loss that is later optimized, so it may bias the reported training performance; using a validation-based selection or reporting the variance across initializations would be more transparent.
- [Section 3.1, Table 1 / Table 2] Tables 1 and 2 appear to contain the same experimental-condition data; the Table 2 caption is incorrect and should be fixed. The paper should also justify the selection of 19 of the original 33 datasets and state the data-quality criteria used.
- [Section 5.1, Eq. (16)] The notation uses X for the state vector, the trajectory matrix, and the library in Eq. (16), which is confusing. Please use distinct symbols, e.g., x, Y, and Theta, for these quantities.
- [References] References [1], [15], and [25] refer to the same work (Chen and Messner) and are formatted inconsistently; there are also other minor citation formatting issues that should be cleaned up.
- [Section 5.2] The statement that the SINDy model 'explicitly describes the dependence of the stress rate on the stress, strain, and (arbitrary) internal state' is unverifiable because the equations are not shown. Please print the full polynomial system.
Circularity Check
Validation is partially built into preprocessing: per-curve normalization in Eq. (12) leaks each validation stress curve's own future extrema into the evaluation, and the SINDy model is fit to the black-box network's trajectories rather than to independent data.
-
fitted input called prediction
[Section 3.1, Eq. (12); evaluation in Section 4, Figs. 9-10 and Fig. 14]
"For the purposes of training the stress-strain curves have been normalized within the range of [-0.5, 0.5]. The normalization equation is given by: x' = (x - min(x))/(max(x)-min(x)) - x0 (12)"
The min and max in Eq. (12) are per-curve extrema of the raw experimental record. The curves are normalized before being split into training and validation sets, so each validation curve's normalization constants are computed from the same measured stress history that the model is supposed to predict. The neural ODE is trained and evaluated on x', meaning the validation target has been rescaled by its own future minimum and maximum. The reported validation MSE therefore measures how well the model reproduces a shape after the true stress amplitude has already been supplied by the preprocessing.
-
fitted input called prediction
[Section 5.1, Eqs. (16)-(18) and the interpretability discussion]
"Upon successfully training a neural ODE model for a black box system, we can readily extract X and its time derivative Ẋ using the trained neural networks. Following this, we can implement symbolic regression with the SINDy algorithm to perform sparse regression and derive the mathematical expression of the dynamic system."
The SINDy model is fit to the state trajectories generated by the already-trained black-box neural ODE, not to independent experimental measurements. The later claim that the interpretable model 'was derived from a black box model... constructed purely from data' is therefore a two-stage fit: the neural ODE was fit to the 19 experimental curves, and SINDy then compressed the neural ODE's own outputs. The reported accuracy 'comparable to the standard Chaboche model' is an in-sample evaluation of this two-stage pipeline, not a discovery validated on withheld experiments. It does not constitute independent confirmation of a newly discovered constitutive equation.
full rationale
The core modeling structure is not circular: the Neural ODE equations are general ansatzes, the Chaboche comparison is externally defined, and the self-citations to pyoptmat are implementation tooling rather than load-bearing evidence for the physics claim. However, the paper's strongest claim—that the neural ODE models 'exceed the standard Chaboche model's accuracy'—rests on a validation comparison made in a per-curve normalized space. Eq. (12) uses each curve's own global minimum and maximum, so for validation curves the stress normalization constants are derived from the very output being predicted. This makes the validation MSE partly a measure of shape reproduction with known extrema, rather than a fully out-of-sample prediction. The conclusion itself concedes this limitation: 'Due to the limited availability of experimental data and the high cost of obtaining additional samples, out-of-sample testing is not feasible for this paper.' In addition, the SINDy interpretable model is fit to the black-box neural ODE's trajectories, not to fresh experimental data, so its 'comparable to Chaboche' accuracy is inherited from a first-stage fit on the same curves. These two issues make the empirical support for the headline claim partially circular, while the methodology itself remains a legitimate modeling approach. Score 6 reflects one or more predictions that reduce, at least in part, to quantities derived from the same data by construction.
Assumptions & free parameters
free parameters (5)
- Chaboche model parameters (E, sigma0, K, eta, n, C_i, gamma_i, delta, R) =
Not reported
- Black-box Neural ODE network weights =
Not reported (3x256 ResNet with GeLU)
- Neural flow rule network weights =
Not reported (same 3x256 ResNet structure)
- SINDy threshold lambda =
0.05
- Per-curve normalization min, max, and offset x0 =
Computed per experiment from full curve
assumptions (5)
- domain assumption A finite set of internal variables h is sufficient to represent the material's loading history (Markovian state assumption).
- domain assumption The 1D model can be extended to 3D using J2 flow theory with a scalar effective stress and effective plastic strain.
- domain assumption The selected 19 of 33 available datasets are a representative subset after data quality exclusions.
- domain assumption Hooke's law provides a valid decomposition for the Neural flow rule model.
- domain assumption The pyoptmat solver and adjoint method correctly compute gradients for the stiff Neural ODE training.
invented entities (1)
-
Arbitrary internal state variables h (and x1, x2 in the SINDy model)
Cite this review
Pith. "Pith review of A Data-Driven Method for Modeling Creep-Fatigue Stress-Strain Behavior Using Neural ODEs." pith.science (2026). https://pith.science/paper/UW4KUUWR
@misc{pith2026250503021,
author = {Pith},
title = {Pith review of: A Data-Driven Method for Modeling Creep-Fatigue Stress-Strain Behavior Using Neural ODEs},
year = {2026},
howpublished = {\url{https://pith.science/paper/UW4KUUWR}},
note = {Machine review of arXiv:2505.03021}
}
read the original abstract
In this paper, we introduce a data-driven machine learning approach for modeling one-dimensional stress-strain behavior under cyclic loading, utilizing experimental data from the nickel-based Alloy 617. The study employs uniaxial creep-fatigue test data acquired under various loading histories and compares two distinct neural network-based ODE models. The first model, known as the black-box model, comprehensively describes the strain-stress relationship using a Neural ODE equation. To interpret this black-box model, we apply the Sparse Identification of Nonlinear Dynamical Systems (SINDy) technique, transforming the black-box model into an equation-based model using symbolic regression. The second model, the Neural flow rule model, incorporates Hooke's Law for the linear elastic component, with the nonlinear part characterized by a Neural ODE. Both models are trained with experimental data to accurately reflect the observed stress-strain behavior. We conduct a detailed comparison with the standard Chaboche model, which includes three back stresses. Our results demonstrate that the neural network-based ODE models precisely capture the experimental creep-fatigue mechanical behavior, exceeding the standard Chaboche model's accuracy. Furthermore, an interpretable model derived from the black-box neural ODE model through symbolic regression achieves accuracy comparable to the Chaboche model, enhancing its interpretability. The results highlight the potential of neural network-based ODE models to depict complex creep-fatigue behavior, eliminating the necessity for experts to define a specific, material-focused model form.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Training material models using gradient descent algorithms
Chen, Tianju, and Mark C. Messner. "Training material models using gradient descent algorithms." International Journal of Plasticity 165 (2023): 103605
work page 2023
-
[2]
An advancement in cyclic plasticity modeling for multiaxial ratcheting simulation,
S. Bari and T. Hassan, "An advancement in cyclic plasticity modeling for multiaxial ratcheting simulation," International Journal of Plasticity, vol. 18, no. 7, pp. 873-894 %@ 0749-6419, 2002
work page 2002
-
[3]
Anatomy of coupled constitutive models for ratcheting simulation,
S. Bari and T. Hassan, "Anatomy of coupled constitutive models for ratcheting simulation," International Journal of Plasticity, vol. 16, no. 3-4, pp. 381-409 %@ 0749-6419, 2000
work page 2000
-
[4]
Implicit constitutive modelling for viscoplasticity using neural networks,
T. Furukawa and G. Yagawa, "Implicit constitutive modelling for viscoplasticity using neural networks," International Journal for Numerical Methods in Engineering, vol. 43, no. 2, pp. 195 - 219 %@ 0029-5981, 1998
work page 1998
-
[5]
L. Benabou, "Development of LSTM networks for predicting viscoplasticity with effects of deformation, strain rate, and temperature history," Journal of Applied Mechanics, vol. 88, no. 7 %@ 0021-8936, 2021
work page 2021
-
[6]
Numerical characterisation of uncured elastomers by a neural network based approach,
C. Zopf and M. Kaliske, "Numerical characterisation of uncured elastomers by a neural network based approach," Computers & Structures, vol. 182, pp. 504-525 %@ 0045-7949, 2017
work page 2017
-
[7]
Deep learning predicts path-dependent plasticity,
M. Mozaffar, R. Bostanabad, W. Chen, K. Ehmann, J. Cao, and M. A. Bessa, "Deep learning predicts path-dependent plasticity," Proceedings of the National Academy of Sciences, vol. 116, no. 52, pp. 26414-26420 %@ 0027-8424, 2019
work page 2019
-
[8]
R. I. Borja, Plasticity. Springer, 2013
work page 2013
Show all 32 references
-
[9]
Elastoplastic constitutive modeling under the complex loading driven by GRU and small-amount data,
Z. Yu, C. Han, H. Yang, Y. Wang, S. Tang, and X. Guo, "Elastoplastic constitutive modeling under the complex loading driven by GRU and small-amount data," Theoretical and Applied Mechanics Letters, vol. 12, no. 6, pp. 100363 %@ 2095-0349, 2022
2022
-
[10]
Deep residual learning for image recognition,
K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," 2016, pp. 770-778
2016
-
[11]
Neural ordinary differential equations,
R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, "Neural ordinary differential equations," Advances in neural information processing systems, vol. 31, 2018
2018
-
[12]
Equivariant flows: sampling configurations for multi -body systems with symmetric energies,
J. Köhler, L. Klein, and F. Noé, "Equivariant flows: sampling configurations for multi -body systems with symmetric energies," arXiv preprint arXiv:1910.00753, 2019
1910 arXiv
-
[13]
Latent ordinary differential equations for irregularly-sampled time series,
Y. Rubanova, R. T. Q. Chen, and D. K. Duvenaud, "Latent ordinary differential equations for irregularly-sampled time series," Advances in neural information processing systems, vol. 32, 2019
2019
-
[14]
Scalable gradients and variational inference for stochastic differential equations,
X. Li, T. -K. L. Wong, R. T. Q. Chen, and D. K. Duvenaud, "Scalable gradients and variational inference for stochastic differential equations," 2020: PMLR, pp. 1-28 %@ 2640-3498
2020
-
[15]
Training material models using gradient descent algorithms,
T. Chen and M. C. Messner, "Training material models using gradient descent algorithms," International Journal of Plasticity, pp. 103605 %@ 0749-6419, 2023
2023
-
[16]
Automatic differentiation in pytorch,
A. Paszke et al., "Automatic differentiation in pytorch," 2017
2017
-
[17]
Hierarchical deep -learning neural networks: finite elements and beyond,
L. Zhang et al. , "Hierarchical deep -learning neural networks: finite elements and beyond," Computational Mechanics, vol. 67, pp. 207-230, 2021
2021
-
[18]
HiDeNN-FEM: a seamless machine learning approach to nonlinear finite element analysis,
Y. Liu, C. Park, Y. Lu, S. Mojumder, W. K. Liu, and D. Qian, "HiDeNN-FEM: a seamless machine learning approach to nonlinear finite element analysis," Computational Mechanics, pp. 1-22, 2023
2023
-
[19]
Convolution Hierarchical Deep-Learning Neural Network Tensor Decomposition (C- HiDeNN-TD) for high -resolution topology optimization,
H. Li et al., "Convolution Hierarchical Deep-Learning Neural Network Tensor Decomposition (C- HiDeNN-TD) for high -resolution topology optimization," Computational Mechanics, pp. 1 -20, 2023
2023
-
[20]
Semi -supervised invertible neural operators for Bayesian inverse problems,
S. Kaltenbach, P. Perdikaris, and P. -S. Koutsourelakis, "Semi -supervised invertible neural operators for Bayesian inverse problems," Computational Mechanics, pp. 1-20, 2023
2023
-
[21]
Information processing, data inferences, and scientific generalization,
D. Gerwin, "Information processing, data inferences, and scientific generalization," Behavioral Science, vol. 19, no. 5, pp. 314-325, 1974
1974
-
[22]
BACON. 5: The discovery of conservation laws,
P. Langley, G. L. Bradshaw, and H. A. Simon, "BACON. 5: The discovery of conservation laws," in IJCAI, 1981, vol. 81: Citeseer, pp. 121-126
1981
-
[23]
Genetic programming as a means for programming computers by natural selection,
J. R. Koza, "Genetic programming as a means for programming computers by natural selection," Statistics and computing, vol. 4, pp. 87-112, 1994
1994
-
[24]
On the limited memory BFGS method for large scale optimization,
D. C. Liu and J. Nocedal, "On the limited memory BFGS method for large scale optimization," Mathematical programming, vol. 45, no. 1-3, pp. 503-528 %@ 0025-5610, 1989
1989
-
[25]
Training material models using gradient descent algorithms,
T. Chen and M. C. Messner, "Training material models using gradient descent algorithms," International Journal of Plasticity, vol. 165, pp. 103605 %@ 0749-6419, 2023
2023
-
[26]
Identity mappings in deep residual networks,
K. He, X. Zhang, S. Ren, and J. Sun, "Identity mappings in deep residual networks," 2016: Springer, pp. 630-645 %@ 3319464922
2016
-
[27]
Delving deep into rectifiers: Surpassing human -level performance on imagenet classification,
K. He, X. Zhang, S. Ren, and J. Sun, "Delving deep into rectifiers: Surpassing human -level performance on imagenet classification," 2015, pp. 1026-1034
2015
-
[28]
Reference constitutive model for Alloy 617 and 316H stainless steel for use with the ASME Division 5 design by inelastic analysis rules,
M. C. Messner and T. L. Sham, "Reference constitutive model for Alloy 617 and 316H stainless steel for use with the ASME Division 5 design by inelastic analysis rules," ed: Argonne National Lab.(ANL), Argonne, IL (United States), 2021
2021
-
[29]
Gaussian error linear units (gelus),
D. Hendrycks and K. Gimpel, "Gaussian error linear units (gelus)," arXiv preprint arXiv:1606.08415, 2016
2016 arXiv
-
[30]
Adam: A method for stochastic optimization,
D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014
2014 arXiv
-
[31]
Discovering governing equations from data by sparse identification of nonlinear dynamical systems,
S. L. Brunton, J. L. Proctor, and J. N. Kutz, "Discovering governing equations from data by sparse identification of nonlinear dynamical systems," Proceedings of the national academy of sciences, vol. 113, no. 15, pp. 3932-3937, 2016
2016
-
[32]
Pysindy: a python package for the sparse identification of nonlinear dynamics from data,
B. M. de Silva, K. Champion, M. Quade, J.-C. Loiseau, J. N. Kutz, and S. L. Brunton, "Pysindy: a python package for the sparse identification of nonlinear dynamics from data," arXiv preprint arXiv:2004.08424, 2020
2004 arXiv
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.