Chaos-Free Networks are Stable Recurrent Neural Networks

Davide Previtali; Fabio Previdi; Mirko Mazzoleni; Stefano De Carli

REVIEW 3 major objections 4 minor 19 references

Chaos-Free Networks are Stable Recurrent Neural Networks

T0 review · 3 major / 4 minor · reviewed 2026-07-14 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A simple gate change makes chaos-free RNNs unconditionally incrementally stable for system ID.

desk verdict Clean unconditional δISS for a simple CFN variant; the math holds and the modeling trade-off is modest. read the letter →

arxiv 2603.14106 v2 pith:SG566GBL submitted 2026-03-14 math.OC cs.SYeess.SY

Stefano De Carli , Davide Previtali , Mirko Mazzoleni , Fabio Previdi This is my paper

classification math.OCcs.SYeess.SY MSC 93D2593B3068T07

keywords Chaos-FreeNetworkDecoupled-GateInput-to-StateStabilityIncrementalISSgatedRNNsnonlinearsystemidentificationbydesign

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Standard gated RNNs can model nonlinear systems well but often produce hard-to-analyze chaotic dynamics and lack built-in stability guarantees. This paper shows that the Chaos-Free Network already satisfies Input-to-State Stability by construction. Full Incremental Input-to-State Stability still needs weight constraints, so the authors introduce the Decoupled-Gate Network: the same architecture with recurrent connections removed from the gates. That structural change alone makes the network unconditionally incrementally stable, so it can be trained with ordinary least-squares loss and still be used for safety-critical identification and control. On two classic benchmarks the new model matches the accuracy of carefully constrained stable LSTMs and GRUs while needing no penalty terms or post-training checks.

What carries the argument

The Decoupled-Gate Network (DGN): a CFN layer whose forget and input gates depend only on the current layer input. With recurrent gate weights zero, the contraction factor of every layer collapses to a number strictly less than one, so the cascade is δISS by construction.

What would settle it

Train a multi-layer DGN on either benchmark, then drive two copies from different initial states with two nearby input sequences; if the state difference fails to decay at a rate bounded by a class-KL function plus a class-K gain on the input difference, the unconditional δISS claim is false.

Watch

Extended reading notes

Core claim

The Chaos-Free Network is ISS by design. Setting its recurrent gate matrices to zero yields the Decoupled-Gate Network, which is unconditionally δISS on the natural forward-invariant set with respect to normalized inputs, without any further parametric restrictions.

Load-bearing premise

After normalization every network input must stay inside the unit hypercube; all the explicit stability gains rest on that compact set together with the fixed ranges of the sigmoid and tanh activations.

Editorial extensions

If this is right

System-identification models can be trained with ordinary MSE loss and still be guaranteed δISS without penalty terms or constrained optimizers.
The same models can be dropped into existing robust MPC or observer schemes that require incremental stability.
Depth recovers expressive power that is lost by removing single-layer gate recurrence.
Any post-training verification of weight-dependent stability conditions becomes unnecessary for DGN architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because δISS is obtained by architecture rather than by optimization, the same construction should transfer immediately to continuous-time or hybrid gated models once the discrete-time cascade argument is rewritten.
Removing gate recurrence may simplify parallel training and hardware mapping of deep RNNs while preserving the stability certificate needed for closed-loop use.
The forward-invariant cube [-2,2] together with the unit-input assumption suggests that simple rescaling of activation ranges could enlarge the admissible operating region without destroying the proofs.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The useful takeaway is Theorem 5: set the recurrent gate matrices of a CFN to zero and you get an L-layer network that is δISS by design on the usual forward-invariant box, with no weight constraints and no penalty terms. That is the new piece. CFN itself is 2016 work; what this paper adds is the first ISS/δISS analysis of it, the observation that the recurrent gates block unconditional δISS, and the DGN structural fix that removes them.

The proofs are standard discrete-time cascade arguments and look solid. Single-layer ISS follows from the contraction of the forget gate and the geometric series; the δISS condition is the usual ρ < 1 built from the 1/4-Lipschitz constant of σ and the 1-Lipschitz constant of φ; the multi-layer case is a lower-triangular interconnection whose spectral radius is controlled by the layer-wise ρ’s. Setting the recurrent matrices to zero collapses ρ to σ-bar_f ∈ (0,1), so the cascade is automatically Schur. Assumption 1 (inputs in [-1,1] after normalization) is explicit and ordinary for the domain; if it is violated the explicit gains change, but that is not a hidden gap.

Numerically they match the usual pH and four-tank benchmarks with competitive Fits (mid-90s) and no stability penalties. The four-tank comparison is not on identical data, and there is no code release, but those are ordinary limitations, not load-bearing flaws. The modeling price of decoupling the gates is small and acknowledged.

This is for people who need δISS RNNs for identification or robust MPC and are tired of penalty terms or post-hoc verification. The citation pattern is appropriate (Jiang-Wang, Bayer et al., the original CFN paper, the recent δISS-LSTM/GRU line). I would send it to referees; the central claim is verifiable mathematics and the engineering motivation is clear. Worth a look if you work in this corner.

Referee Report

3 major / 4 minor

Summary. The paper analyzes the Chaos-Free Network (CFN) of Laurent & von Brecht through the lens of discrete-time ISS and incremental ISS (δISS). It proves that every CFN layer (and therefore any cascade of layers) is ISS on the forward-invariant set H_inv = [-2,2]^{n_h} with respect to normalized inputs (Theorems 1 and 3). δISS of a CFN layer holds only under an explicit contraction condition ρ^{(l)} < 1 that involves the recurrent gate matrices (Theorem 2); the same condition must hold layer-wise for the deep network (Theorem 4). The authors then introduce the Decoupled-Gate Network (DGN) by simply setting those recurrent gate matrices to zero. Because the resulting contraction factor collapses to the open interval (0,1), the DGN is unconditionally δISS (Theorem 5). Two standard system-identification benchmarks (pH neutralization and quadruple tank) show that single- and multi-layer DGNs attain Fit values competitive with previously published δISS-LSTM/GRU models that required stability-augmented losses.

Significance. If the proofs hold, the work supplies a gated recurrent neural architecture that is δISS by design, without weight constraints, penalty terms, or post-training verification. That property is directly useful for observer design and robust MPC, where incremental stability is a standing hypothesis. The derivations reuse only standard Lipschitz constants of σ and ϕ, geometric-series bounds, and the classical cascade theorem for ISS systems; they are therefore transparent and readily checkable. The numerical section further shows that the structural simplification does not destroy modeling accuracy on two well-known nonlinear benchmarks, giving a concrete existence proof that unconditional δISS need not be purchased at a large price in Fit. These features make the contribution of clear interest to the learning-based control community.

major comments (3)

Section V.C (Four-Tank): the δISS-GRU baseline of Bonassi et al. (2021) was trained and tested on a different realization of the same plant. Consequently the reported Fit gap (97.0 % versus 93.4–95.1 %) cannot be interpreted as a controlled comparison of architectures. Either retrain the baseline on the exact data splits used for CFN/DGN or remove the quantitative claim of “comparable performance” relative to that specific number.
Section V.A and Theorems 2/4: the trained CFN models are stated to violate the sufficient condition ρ^{(l)} < 1, yet no numerical check is performed to determine whether those models are nevertheless δISS (or exhibit incremental instability). Because the paper’s central selling point is that the DGN removes the need for such conditions, it is important to clarify how conservative (6) actually is; a short Monte-Carlo experiment on the trained CFNs would settle the question.
Definition 1: the ISS estimate includes an additive class-K_∞ term γ_b(||b||) that treats the bias vector as an exogenous input. While the subsequent proofs remain correct under this definition, the formulation is non-standard relative to the classical discrete-time ISS literature (Jiang & Wang, 2001) and to the δISS definition used later in the paper. A short remark explaining why the bias is singled out (or a reduction to the usual ISS statement by absorbing constant biases into an extended input) would avoid confusion.

minor comments (4)

Figure 1 caption: the dashed paths are said to be “recurrent connections in the gates”; it would help the reader to mark explicitly which of those paths are removed in the DGN.
Equation (8): the washout length N_w = 25 is fixed for both benchmarks; a one-sentence justification (or a sensitivity check) would be useful.
References [5] and [6] already contain the cascade argument used in the proof of Theorem 4; a more explicit pointer to the relevant appendix of those papers would improve traceability.
Typographical: “non-divergent but non-vanishing oscillations” (p. 1) and the repeated phrase “by design” could be tightened for concision.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ISS/δISS claims follow by direct bounds on σ/ϕ ranges, 1/4-Lipschitz of sigmoid, cascade of ISS systems, and the structural definition R_f=R_i=0, none of which reduce to fitted quantities or load-bearing self-citations.

full rationale

The derivation chain is self-contained. Proposition 1 obtains the forward-invariant set H_inv=[-2,2]^{n_h} solely from the ranges σ∈(0,1) and ϕ∈(-1,1). Theorem 1 produces the ISS KL/K∞ gains by the same range bound on the forget gate (¯σ_f∈(0,1)) plus 1-Lipschitz of tanh, yielding a geometric series that is independent of any data fit. Theorem 2 obtains the single-layer contraction factor ρ=¯σ_f+(1/4)∥R_f∥+(1/4)∥R_i∥¯ϕ_˜h by the identical Lipschitz constants; the condition ρ<1 is an explicit parametric hypothesis, not a fitted prediction. Theorems 3–4 lift the layer results to the cascade via the standard ISS cascade theorem of Jiang & Wang and the lower-triangular Schur argument of Bayer et al. (with a non-self citation to Bonassi et al. for the multi-layer unwrapping). Theorem 5 simply substitutes the architectural definition R_f^{(l)}=R_i^{(l)}=0 into (6), which forces ρ^{(l)}=¯σ_f^{(l)}∈(0,1) by the range of σ; the resulting A_δ is Schur by construction of the architecture, not by circular appeal to the target property. Numerical experiments train free weights by ordinary MSE and never feed the resulting parameters back into the theorems. The single self-citation [8] appears only as an analogy for prior constrained-training practice and is not used in any proof step. Consequently no step reduces a claimed prediction or first-principles result to its own inputs.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The theoretical claims rest on standard comparison-function definitions of ISS/δISS, the known cascade theorem for ISS systems, elementary Lipschitz bounds on σ and ϕ, and the standing normalization Assumption 1. No free parameters are fitted inside the stability proofs; network weights are free only for the empirical modeling experiments. The sole invented entity is the DGN architecture itself, introduced precisely to obtain unconditional δISS.

free parameters (3)

hidden units per layer n_h^(l) = 7
Chosen by hand to match published baselines (n_h=7); affects empirical Fit but not the δISS theorems.
network depth L = 1 or 3
Set to 1 (pH) or 3 (four-tank) to match baselines; depth is required for DGN expressivity but is not a fitted constant inside the proofs.
Adam learning-rate schedule and dropout = 0.001 / 0.05
Base lr 0.001, decay 0.9 every 200 epochs, dropout 0.05; ordinary training hyperparameters that do not enter the stability statements.

assumptions (5)

standard math Definitions of ISS and δISS via class-KL and class-K∞ comparison functions (Defs. 1–2, citing Jiang & Wang 2001, Bayer et al. 2013).
Taken as background; all theorems are stated with respect to these definitions.
standard math Cascade of ISS systems is ISS (invoked in Thm 3 proof, citing Jiang & Wang).
Used to lift single-layer ISS to the multi-layer CFN/DGN.
standard math σ is 1/4-Lipschitz and ϕ is 1-Lipschitz; ranges σ∈(0,1), ϕ∈(−1,1).
Elementary calculus facts used for all gate and state bounds (Props. 1, Thms 1–2).
domain assumption Assumption 1: inputs lie in the compact set U=[−1,1]^{n_u} after normalization.
Stated explicitly in Section IV; every explicit gain function is derived from this bound.
ad hoc to paper Forward-invariant set H_inv=[−2,2]^{n_h} for each layer (Prop. 1).
Derived from activation ranges but fixed as the working state space for all subsequent theorems.

invented entities (1)

Decoupled-Gate Network (DGN)
purpose: Structural variant of CFN obtained by setting recurrent gate matrices R_f=R_i=0 so that δISS holds unconditionally.
Introduced in Section III-B; no prior independent evidence outside this paper. Empirical modeling performance is offered as supporting evidence but is not a falsifiable external prediction.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Chaos-Free Networks are Stable Recurrent Neural Networks." pith.science (2026). https://pith.science/paper/SG566GBL

@misc{pith2026260314106,
  author       = {Pith},
  title        = {Pith review of: Chaos-Free Networks are Stable Recurrent Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SG566GBL}},
  note         = {Machine review of arXiv:2603.14106}
}

read the original abstract

Gated Recurrent Neural Networks (RNNs) are widely used for nonlinear system identification due to their high accuracy, although they often exhibit complex, chaotic dynamics that are difficult to analyze. This paper investigates the system-theoretic properties of the Chaos-Free Network (CFN), an architecture originally proposed to eliminate the chaotic behavior found in standard gated RNNs. First, we formally prove that the CFN satisfies Input-to-State Stability (ISS) by design. However, we demonstrate that the CFN architecture does not intrinsically guarantee Incremental ISS (delta-ISS), as ensuring this property relies on specific parametric constraints. To address this, we introduce the Decoupled-Gate Network (DGN), a novel structural variant of the CFN that removes internal state connections in the gating mechanisms. Finally, we prove that the DGN unconditionally satisfies the delta-ISS property, providing an incrementally stable architecture for identifying nonlinear dynamical systems without requiring complex network training modifications. Numerical results confirm that the DGN maintains the modeling capabilities of standard architectures while adhering to these rigorous stability guarantees.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

19 extracted references · 2 linked inside Pith

[1]

Deep networks for system identification,

G. Pillonetto, A. Aravkin, D. Gedon, L. Ljung, A. H. Ribeiro, and T. B. Sch ¨on, “Deep networks for system identification,”Automatica, vol. 171, p. 111907, 2025

2025
[2]

K. P. Murphy,Probabilistic Machine Learning. The MIT Press, 2022

2022
[3]

A recurrent neural network without chaos,

T. Laurent and J. von Brecht, “A recurrent neural network without chaos,” 2016. [Online]. Available: https://arxiv.org/abs/1612.06212

arXiv 2016
[4]

On Recurrent Neural Networks for learning-based control,

F. Bonassi, M. Farina, J. Xie, and R. Scattolini, “On Recurrent Neural Networks for learning-based control,”J. Process Control, vol. 114, pp. 92–104, 2022

2022
[5]

Learning model predictive control with long short-term memory networks,

E. Terzi, F. Bonassi, M. Farina, and R. Scattolini, “Learning model predictive control with long short-term memory networks,”Int. J. Robust Nonlin. Control, vol. 31, no. 18, pp. 8877–8896, 2021

2021
[6]

On the stability properties of Gated Recurrent Units neural networks,

F. Bonassi, M. Farina, and R. Scattolini, “On the stability properties of Gated Recurrent Units neural networks,”Syst. Control Lett., vol. 157, p. 105049, 2021

2021
[7]

An Incremental Input-to- State Stability Condition for a Class of Recurrent Neural Networks,

W. D’Amico, A. La Bella, and M. Farina, “An Incremental Input-to- State Stability Condition for a Class of Recurrent Neural Networks,” IEEE Trans. Autom. Control, vol. 69, no. 4, pp. 2221–2236, 2024

2024
[8]

Infinity-norm-based Input-to-State-Stable Long Short- Term Memory networks,

S. De Carli, D. Previtali, L. Pitturelli, M. Mazzoleni, A. Ferramosca, and F. Previdi, “Infinity-norm-based Input-to-State-Stable Long Short- Term Memory networks,” in2025 European Control Conference (ECC), 2025, pp. 911–916

2025

Show all 19 references

[9]

Robust Offset-Free Constrained Model Predictive Control With Long Short-Term Memory Networks,

I. Schimperna and L. Magni, “Robust Offset-Free Constrained Model Predictive Control With Long Short-Term Memory Networks,”IEEE Trans. Automat. Contr., vol. 69, no. 12, pp. 8172–8187, 2024

2024
[10]

Robust constrained nonlinear Model Predictive Control with Gated Recurrent Unit model,

——, “Robust constrained nonlinear Model Predictive Control with Gated Recurrent Unit model,”Automatica, vol. 161, p. 111472, 2024

2024
[11]

Stable Recurrent Models,

J. Miller and M. Hardt, “Stable Recurrent Models,” 2018. [Online]. Available: https://arxiv.org/abs/1805.10369

2018 arXiv
[12]

Deep equilibrium models,

S. Bai, J. Z. Kolter, and V . Koltun, “Deep equilibrium models,” in Advances in Neural Information Processing Systems (NeurIPS), 2019

2019
[13]

Strogatz,Nonlinear Dynamics and Chaos, ser

S. Strogatz,Nonlinear Dynamics and Chaos, ser. A Chapman & Hall Book. CRC Press, 2019

2019
[14]

A compendium of comparison function results,

C. M. Kellett, “A compendium of comparison function results,” Mathematics of Control, Signals, and Systems, vol. 26, no. 3, pp. 339– 374, 2014

2014
[15]

Input-to-state stability for discrete-time nonlinear systems,

Z.-P. Jiang and Y . Wang, “Input-to-state stability for discrete-time nonlinear systems,”Automatica, vol. 37, no. 6, pp. 857–869, 2001

2001
[16]

Discrete-time Incremental ISS,

F. Bayer, M. Burger, and F. Allgower, “Discrete-time Incremental ISS,” in2013 European Control Conference (ECC). IEEE, 2013, pp. 2068– 2073

2013
[17]

Goodfellow, Y

I. Goodfellow, Y . Bengio, and A. Courville,Deep Learning. The MIT press, 2016

2016
[18]

Were RNNs All We Needed?

L. Feng, F. Tung, M. O. Ahmed, Y . Bengio, and H. Hajimirsadegh, “Were RNNs All We Needed?” 2024

2024
[19]

C. D. Meyer,Matrix analysis and applied linear algebra. SIAM, 2023

2023

Pith tools

Reviewed July 14, 2026 · model on record in the stance chip above.

[1] [1]

Deep networks for system identification,

G. Pillonetto, A. Aravkin, D. Gedon, L. Ljung, A. H. Ribeiro, and T. B. Sch ¨on, “Deep networks for system identification,”Automatica, vol. 171, p. 111907, 2025

2025

[2] [2]

K. P. Murphy,Probabilistic Machine Learning. The MIT Press, 2022

2022

[3] [3]

A recurrent neural network without chaos,

T. Laurent and J. von Brecht, “A recurrent neural network without chaos,” 2016. [Online]. Available: https://arxiv.org/abs/1612.06212

arXiv 2016

[4] [4]

On Recurrent Neural Networks for learning-based control,

F. Bonassi, M. Farina, J. Xie, and R. Scattolini, “On Recurrent Neural Networks for learning-based control,”J. Process Control, vol. 114, pp. 92–104, 2022

2022

[5] [5]

Learning model predictive control with long short-term memory networks,

E. Terzi, F. Bonassi, M. Farina, and R. Scattolini, “Learning model predictive control with long short-term memory networks,”Int. J. Robust Nonlin. Control, vol. 31, no. 18, pp. 8877–8896, 2021

2021

[6] [6]

On the stability properties of Gated Recurrent Units neural networks,

F. Bonassi, M. Farina, and R. Scattolini, “On the stability properties of Gated Recurrent Units neural networks,”Syst. Control Lett., vol. 157, p. 105049, 2021

2021

[7] [7]

An Incremental Input-to- State Stability Condition for a Class of Recurrent Neural Networks,

W. D’Amico, A. La Bella, and M. Farina, “An Incremental Input-to- State Stability Condition for a Class of Recurrent Neural Networks,” IEEE Trans. Autom. Control, vol. 69, no. 4, pp. 2221–2236, 2024

2024

[8] [8]

Infinity-norm-based Input-to-State-Stable Long Short- Term Memory networks,

S. De Carli, D. Previtali, L. Pitturelli, M. Mazzoleni, A. Ferramosca, and F. Previdi, “Infinity-norm-based Input-to-State-Stable Long Short- Term Memory networks,” in2025 European Control Conference (ECC), 2025, pp. 911–916

2025