REVIEW 3 major objections 4 minor 38 references
On Kernel Design for Regularized Volterra Series Identification of Wiener-Hammerstein Systems
T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Structured kernels built from the Wiener-Hammerstein block prior identify Volterra series in cubic time and, for separable inputs, near-linear time.
desk verdict A genuinely new structural kernel design for Volterra identification, with real complexity gains, but the central PSD theorem has a fixable gap on negative lags. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the factored Volterra-map identity $h_m(t_1,\ldots,t_m)=a_m\sum_{\tau=0}^{n-1}g_2(\tau)\prod_{i=1}^m g_1(t_i-\tau)$, which expresses every order of a Wiener-Hammerstein system as a shared convolution with $g_2$ of products of $g_1$. The designed kernel (10a)-(10b) replaces $g_1,g_2$ by kernels $\kappa_1,\kappa_2$ and $g_1$ by $\zeta$; Theorem 3.1 guarantees the full kernel matrix is positive semidefinite exactly when $\kappa_1(t,s)-\zeta(t)\zeta(s)$ is positive semidefinite. Computation is carried by two structural identities: $Q=\mathrm{conv2}(K_2,Q_w)$ for the output kernel, and, in the Wiener special case, the separability decomposition $Q=\bar{U}\bar{V}^T$ whose rank $\gamma$ is given by the combinatorial formula in Theorem 4.2.
What would settle it
For the Wiener special case $\kappa_2(t,s)=1$ when $t=s=0$ and $0$ otherwise, take input $u(t)=e^{\lambda t}$ (so $r=1$) and $M=3$; Theorem 4.2 says the output kernel matrix $Q_w$ has exact separability rank $\gamma=5$. Compute the singular values of the numerically formed $Q_w$ for $N$ large: if the sixth singular value is not at machine precision relative to the largest, the rank formula fails. The same check applies for $u(t)=e^{-0.0003t}\cos(0.1t+\pi/3)$ with $r=2$ and $M=3$, where the predicted rank is $\gamma=14$.
Extended reading notes
Core claim
The central claim is that the optimal-kernel identity $P_{\mathrm{opt}}=\theta_0\theta_0^T$, applied to the exact Volterra-map form of a WH system, produces an explicit cross-kernel $$K_{pq}(t_p,s_q;\eta_{pq})=\sum_{\xi_1=0}^{n-1}\sum_{\xi_2=0}^{n-1}\kappa_2(\xi_1,\xi_2;\eta_2)K^w_{pq}(t_p-\xi_1,s_q-\xi_2;a_p,a_q,\eta_1),$$ where $\kappa_1,\kappa_2$ are user-designed kernels for the two linear blocks and $\zeta$ is a designed stand-in for $g_1$. This kernel has nonzero off-diagonal blocks, so it encodes correlations between different Volterra orders, and it is positive semidefinite whenever $\kappa_1(t,s)-\zeta(t)\zeta(s)$ is positive semidefinite. The output kernel matrix has the convolution form $Q=\mathrm{conv2}(K_2,Q_w)$, computable in $O(N^3)$; in the special case $\kappa_2(t,s)=1$ if $t=s=0$ and $0$ otherwise, with inputs satisfying $u(t-b)=\sum_{i=1}^r\pi_i(t)\rho_i(b)$, $Q$ is exactly separable with rank $\gamma=\frac{(r+M-1)!}{(r-1)!M!}+2\sum_{m=1}^{M-1}\frac{(r+m-1)!}{(r-1)!m!}$, reducing the empirical-Bayes and prediction computations to $O(N\gamma^2)$.
Load-bearing premise
The load-bearing premise is that the true system is exactly a Wiener-Hammerstein system with finite-memory linear blocks of length $n$ and a static nonlinearity that is a polynomial of degree $M$; if the real dynamics have infinite memory or a non-polynomial nonlinearity, the kernel encodes the wrong structure and can introduce bias.
Editorial extensions
If this is right
- For Wiener-Hammerstein systems with polynomial nonlinearity, the regularization kernel can encode block-structure and cross-order correlations rather than only smoothness and decay.
- The empirical-Bayes estimate and hyperparameter criterion can be evaluated in $O(N^3)$ time independent of the number of Volterra coefficients $n_\theta$.
- For Wiener systems with separable input signals, the output kernel matrix has explicit rank $\gamma$ and the computation scales as $O(N\gamma^2)$, i.e., linearly in sample size for fixed $\gamma$.
- On the tested Wiener-Hammerstein and Wiener databanks, the proposed kernels give better average prediction fit than the SED-MPK kernel, and better robustness than Bayesian semiparametric and prediction-error baselines in the D3 comparison.
- Hyperparameters are still tuned by empirical Bayes, preserving automatic model-complexity control in short or low-SNR data.
Reading between the lines
- The same kernel construction should extend to Hammerstein systems and to parallel interconnections of Wiener-Hammerstein blocks, because the Volterra map of a parallel structure is a sum of terms of type (8) and the kernel would be the corresponding sum, keeping the convolution form.
- Because $\gamma$ depends only on $M$ and $r$, the $O(N\gamma^2)$ regime is best for weakly nonlinear systems or low-complexity inputs; a testable extension is to update the separable generators incrementally for online or adaptive identification.
- When the FIR/polynomial prior is wrong, a natural safeguard is to add a generic smoothness kernel term to the designed kernel, giving an estimator that can fall back to smoothness when the block-structure prior is misspecified.
- The comparisons emphasize prediction and block fits; a further useful test is whether the off-diagonal blocks recover the true cross-correlations between Volterra orders, something the block-diagonal baselines cannot do.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses kernel-based regularized identification of Volterra series for Wiener-Hammerstein (WH) systems with polynomial nonlinearity. Starting from the optimal kernel θ_p θ_q^T, the authors design structural kernels K_pq of the form (10a)-(10b), which embed the WH convolution structure through κ2 and products of κ1 and ζ. Theorem 3.1 states a positive semidefiniteness (PSD) condition for the resulting kernel matrix; Theorem 4.1 gives an O(N^3) construction of the output kernel matrix Q via a 2-D convolution; Theorem 4.2 shows that, for a delta-like κ2 and inputs satisfying a separability condition (15), Q has separability rank γ and the cost drops to O(Nγ^2). The paper includes Monte Carlo experiments on four databanks, comparing the proposed kernels against SED-MPK and two existing Wiener-system methods.
Significance. If the proof gaps identified below are fixed, this is a valuable contribution: it provides a direct, structural kernel for Volterra maps of WH systems with nonzero off-diagonal blocks, and it matches the state-of-the-art O(N^3) complexity while encoding more prior knowledge than previous kernels. The separability-rank analysis in Theorem 4.2 is nontrivial and the simulations support the practical usefulness of the approach. The authors provide detailed appendix proofs and a transparent experimental setup, which are strengths. The main weaknesses are the incomplete PSD guarantee for the implemented kernel at negative lags and the underspecified convolution in Theorem 4.1; both appear fixable, which is why I recommend major revision rather than rejection.
major comments (3)
- [Section 3, Eq. (10a), and Theorem 3.1] The proposed kernel evaluates κ1(t_i−ξ1, s_i−ξ2) and ζ at arguments that can be negative whenever ξ1>t_i or ξ2>s_i, since ξ1, ξ2 range over 0,...,n−1 while t_i, s_i start at 0. However, κ1 and ζ are introduced in Section 2.1 only on R_+, and the proof of Theorem 3.1 in Appendix A.2 establishes PSD of P^w only for nonnegative indices. If the DC formulas are naively extended to negative arguments, the sufficient condition of Theorem 3.1 fails: for ζ as in Corollary 3.1(1), κ1(−r,−r)−ζ(−r)^2 = c1^2 e^{2αr}(1−e^{2βr}) < 0 for r>0. Thus the PSD guarantee does not cover the implemented kernel whenever κ2 has nonzero off-diagonal entries, which is exactly the D1/D2 setting. The authors should specify a causal zero extension (or another explicit extension) for κ1, ζ, and Kw at negative lags and then prove that the convolution in (10a) preserves PSD under that extension; as written, the claim that P defined through (10) is PSD is not established.
- [Section 4.1, Eq. (11), and Theorem 4.1] With K2 ∈ R^{n×n} and Qw ∈ R^{N×N}, the 2-dimensional convolution conv2(K2, Qw) as defined in the footnote has size (N+n−1)×(N+n−1), not N×N as stated for Q. Moreover, Eqs. (30)-(32) require Qw_pq(t−ξ1, s−ξ2) at negative indices when t or s is near 1, but Qw in (12) is only defined for indices 1,...,N. The theorem should specify the zero-padding convention (for example, u(t)=0 for t≤0, corresponding to the pre-windowing option in footnote 2) and state which submatrix of the convolution is taken. Without this, the claimed O(N^3) construction of Q is not well defined.
- [Section 5, databanks D1-D3] The structural prior (8) assumes FIR linear blocks G1 and G2 and a polynomial nonlinearity, but Databanks D1 and D2 use IIR linear blocks and Databank D3 uses a saturation nonlinearity. The only justification is the qualitative truncation and polynomial-approximation argument in Remarks 2.1 and 2.2. Since these experiments are used as evidence for the kernel's effectiveness, the paper should either provide a quantitative statement about the resulting approximation error or explicitly present these cases as robustness checks of a misspecified model rather than as instances where the structural prior holds.
minor comments (4)
- [Abstract and Theorem 4.2] The abstract states that the complexity is lowered to O(Nγ^2), but Theorem 4.2 gives O(Nγ^2+n^2r) and requires the additional assumption that κ1 is extended-p semiseparable in order to remove the n^2r term; the abstract should state this extra condition.
- [Corollary 3.1(2) and Appendix A.3] The eigenfunction notation is inconsistent: the corollary statement uses ψ_i while the proof uses ϕ_i for the same objects, and the claim l≪N is not satisfied in the simulations of Databank D2, where l=100 and N=400.
- [Section 5.1] The label 'R VS' is used without definition; it should be spelled out on first use, for example 'regularized Volterra series (R VS)'.
- [Section 4.2, Eq. (15)] It would be useful to state explicitly which test inputs satisfy the separability assumption; white Gaussian noise (D2) does not satisfy it exactly, while the damped cosine of D4 does with small r.
Circularity Check
No significant circularity: kernel (10) is built from the WH structural prior (8) via the standard optimal-kernel plug-in guideline; hyper-parameters are tuned on training data, all fits are computed on held-out test data (Sec. 5.1-5.3); same-author citations [17,33] are non-load-bearing building blocks (the key bound (29) is proven in-paper); no fitted value is relabeled as a prediction.
full rationale
The central claims — kernel (10) encodes the WH structure (8), Theorem 3.1's PSD condition, Theorem 4.1's O(N^3) construction, and Theorem 4.2's separability rank γ with O(Nγ^2) complexity — are derived in the paper rather than imported. Kernel (10) is obtained from Proposition 3.1 by the standard plug-in procedure (replace g1^0, g2^0 by κ1, κ2, ζ), i.e., it is constructed from the model class, not from the data used to evaluate it: Section 5.1 explicitly splits the noiseless data into training and test sets, the EB cost (5) is minimized on training data, and PFit/GFit/NFit (19)-(20) are computed on held-out test data, so no fitted hyper-parameter is relabeled as a prediction. The same-author citations ([17] for Lemma 2.1 and the DC kernel, [28] for the design guideline, [33] for the DC-kernel Mercer expansion, [34]/[38] for semiseparable complexity) do not carry the conclusions: Lemma 2.1 is used only as motivation ('Clearly, P^opt cannot be applied in practice'), and Corollary 3.1(2) proves in-paper the bound ∥λ∥² = 8/π²·Σ_i 1/(2i−1)² < 1 (Eq. 29) on top of the parameter-free eigenfunction expansion of [33]. Remark 4.2 explicitly concedes that the diagonal special case of (10) has an output kernel equivalent to the known SED-MPK [9, Eq. (36)], so no known result is relabeled. The skeptic's negative-lag concern — the A.2 proof covers K^w at nonnegative lags while the convolution in (10a) evaluates κ1, ζ at negative lags, where the DC extension gives κ1(−r,−r)−ζ(−r)² < 0 — is a possible correctness gap in Theorem 3.1, not circularity: it involves no fitted parameter, no self-citation chain, and no definitional equivalence, and it does not affect Theorem 4.1's input-shifting construction or Theorem 4.2's rank algebra. Score 2 reflects several same-author building-block citations that are instrumental but non-load-bearing; the derivation itself is self-contained and benchmarked against external methods ([9,14,22]).
Assumptions & free parameters
free parameters (6)
- DC kernel parameters (c1, α1, β1) for κ1
- DC kernel parameters (c2, α2, β2) for κ2
- Polynomial coefficients a_m, m=0,...,M
- Noise variance σ²
- Number of basis functions l for ζ in Corollary 3.1 case 2 =
100 in D2 and D4 simulations
- Separability rank r of the input representation (15) =
2 for the D4 input u(t)=e^{-0.0003t} cos(0.1t+π/3); otherwise assumed small
assumptions (6)
- domain assumption The true system is exactly a Wiener-Hammerstein system with a polynomial static nonlinearity (7) and FIR linear blocks of order n, giving Volterra maps (8).
- domain assumption The finite Volterra series (1) is a good truncation of the infinite Volterra series, with bias made small by choosing n and M large.
- domain assumption The linear blocks are stable and overdamped so that the DC kernel (6) is an appropriate prior.
- domain assumption The input signal satisfies (15) for Theorem 4.2, and the input is persistently exciting.
- domain assumption Measurement noise v(t) is i.i.d. Gaussian with zero mean and variance σ².
- standard math Standard matrix identities and kernel theory: Schur product theorem, convolution theorem, multinomial theorem, matrix inversion lemma, determinant lemma, and the eigendecomposition of the DC kernel from [33].
Cite this review
Pith. "Pith review of On Kernel Design for Regularized Volterra Series Identification of Wiener-Hammerstein Systems." pith.science (2026). https://pith.science/paper/ZESQ2NWU
@misc{pith2026250520747,
author = {Pith},
title = {Pith review of: On Kernel Design for Regularized Volterra Series Identification of Wiener-Hammerstein Systems},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZESQ2NWU}},
note = {Machine review of arXiv:2505.20747}
}
abstract
There have been increasing interests on the Volterra series identification with the kernel-based regularization method. The major difficulties are on the kernel design and efficiency of the corresponding implementation. In this paper, we first assume that the underlying system to be identified is the Wiener-Hammerstein (WH) system with polynomial nonlinearity. We then show how to design kernels with nonzero off-diagonal blocks for Volterra maps by taking into account the prior knowledge of the linear blocks and the structure of WH systems. Moreover, exploring the structure of the designed kernels leads to the same computational complexity as the state-of-the-art result, i.e., $O(N^3)$, where $N$ is the sample size, but with a significant difference that the proposed kernels are designed in a direct and flexible way. In addition, for a special case of the kernel and a class of widely used input signals, further exploring the separable structure of the output kernel matrix can lower the computational complexity from $O(N^3)$ to $O(N\gamma^2)$, where $\gamma$ is the separability rank of the output kernel matrix and can be much smaller than $N$. We finally run Monte Carlo simulations to demonstrate the proposed kernels and the obtained theoretical results.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
S. Boyd and L. Chua. Fading memory and the problem of approximating nonlinear operators with Volterra series. IEEE Transactions on Circuits and Systems , 32(11):1150– 1161, 1985
work page 1985
-
[2]
G. Pillonetto, F. Dinuzzo, T. Chen, G. De Nicolao, and L. Ljung. Kernel methods in system identification, machine learning and function estimation: A survey. Automatica, 50(3):657–682, 2014
work page 2014
-
[3]
A. Chiuso. Regularization and Bayesian learning in dynamical systems: Past, present and future.Annual Reviews in Control, 41:24–38, 2016
work page 2016
- [4]
-
[5]
G. Pillonetto, T. Chen, A. Chiuso, G. De Nicolao, and L. Ljung. Regularized System Identification: Learning Dynamic Models from Data . Springer Nature, 2022
work page 2022
-
[6]
J. G. Stoddard, J. S. Welsh, and H. Hjalmarsson. EM-based hyperparameter optimization for regularized Volterra kernel estimation. IEEE Control Systems Letters , 1(2):388–393, 2017
work page 2017
-
[7]
G. Birpoutsoukis, A. Marconato, J. Lataire, and J. Schoukens. Regularized nonparametric Volterra kernel estimation. Automatica, 82:324–327, 2017
work page 2017
-
[8]
G. Birpoutsoukis, P. Z. Csurcsia, and J.Schoukens. Efficient multidimensional regularization for Volterra series estimation. Mechanical Systems and Signal Processing , 104:896–914, 2018
work page 2018
Show all 38 references
-
[9]
Dalla Libera, R
A. Dalla Libera, R. Carli, and G. Pillonetto. Kernel- based methods for Volterra series identification. Automatica, 129:109686, 2021
2021
-
[10]
G. Palm. On representation and approximation of nonlinear systems. part ii: Discrete time. Biol. Cybern., 34(1):49 – 52, 1979
1979
-
[11]
Bai and F
E.W. Bai and F. Giri. Block-oriented Nonlinear System Identification. Springer London, London, 2010
2010
-
[12]
Schoukens and L
J. Schoukens and L. Ljung. Nonlinear system identification: A user-oriented road map. IEEE Control Systems Magazine, 39(6):28–99, 2019
2019
-
[13]
Enqvist and L
M. Enqvist and L. Ljung. Linear approximations of nonlinear FIR systems for separable input processes. Automatica, 41(3):459–473, 2005. Data-Based Modelling and System Identification
2005
-
[14]
Hagenblad, L
A. Hagenblad, L. Ljung, and A. Wills. Maximum likelihood identification of Wiener models. Automatica, 44(11):2697– 2705, 2008
2008
-
[15]
Tiels and J
K. Tiels and J. Schoukens. Wiener system identification with generalized orthonormal basis functions. Automatica, 50(12):3147–3154, 2014
2014
-
[16]
Schoukens and K
M. Schoukens and K. Tiels. Identification of block-oriented nonlinear systems starting from linear approximations: A survey. Automatica, 85:272–292, 2017
2017
-
[17]
T. Chen, H. Ohlsson, and L. Ljung. On the estimation of transfer functions, regularizations and Gaussian processes- revisited. Automatica, 48(8):1525–1535, 2012
2012
-
[18]
Pillonetto and A
G. Pillonetto and A. Chiuso. Tuning complexity in regularized kernel-based regression and linear system identification: The robustness of the marginal likelihood estimator. Automatica, 58:106–117, 2015. 16
2015
-
[19]
B. Mu, T. Chen, and L. Ljung. On asymptotic properties of hyperparameter estimators for kernel-based regularization methods. Automatica, 94:381–395, 2018
2018
-
[20]
Y. Ju, B. Mu, L. Ljung, and T. Chen. Asymptotic theory for regularized system identification part I: Empirical Bayes hyperparameter estimator. IEEE Transactions on Automatic Control, 68(12):7224–7239, 2023
2023
-
[21]
Pillonetto and A
G. Pillonetto and A. Chiuso. Gaussian processes for Wiener- Hammerstein system identification. IF AC Proceedings Volumes, 42(10):838–843, 2009. 15th IF AC Symposium on System Identification
2009
-
[22]
Lindsten, T
F. Lindsten, T. B. Sch¨ on, and M. I. Jordan. Bayesian semiparametric Wiener system identification. Automatica, 49(7):2053–2063, 2013
2013
-
[23]
R. S. Risuleo, G. Bottegal, and H. Hjalmarsson. A nonparametric kernel-based approach to Hammerstein system identification. Automatica, 85:234–247, 2017
2017
-
[24]
R. S. Risuleo, F. Lindsten, and H. Hjalmarsson. Bayesian nonparametric identification of Wiener systems. Automatica, 108:108480, 2019
2019
-
[25]
N. Wiener. Nonlinear problems in random theory . Wiley, New York, 1958
1958
-
[26]
Schetzen
M. Schetzen. The Volterra and Wiener Theories of Nonlinear Systems. John Wiley & Sons, 1980
1980
-
[27]
L. Ljung. System identification: theory for the user . Upper Saddle River, NJ: Prentice Hall, 12 1999
1999
-
[28]
T. Chen. On kernel design for regularized LTI system identification. Automatica, 90:109–122, 2018
2018
-
[29]
Schoukens, R
J. Schoukens, R. Pintelon, and M. Enqvist. Study of the LTI relations between the outputs of two coupled Wiener systems and its application to the generation of initial estimates for Wiener–Hammerstein systems. Automatica, 44(7):1654– 1665, 2008
2008
-
[30]
Nowak and B.D
R.D. Nowak and B.D. Van Veen. Random and pseudorandom inputs for Volterra filter identification. IEEE Transactions on Signal Processing, 42(8):2124–2135, 1994
1994
-
[31]
Birpoutsoukis
G. Birpoutsoukis. Volterra series estimation in the presence of prior knowledge . PhD thesis, Vrije Universiteit Brussel Brussel, Belgium, 2018
2018
-
[32]
W. K. Wong. Comparing robust properties of A, D, E and G- optimal designs. Computational Statistics & Data Analysis , 18(4):441–448, 1994
1994
-
[33]
T. Chen. Continuous-time DC kernel—a stable generalized first-order spline kernel. IEEE Transactions on Automatic Control, 63(12):4442–4447, 2018
2018
-
[34]
Chen and M
T. Chen and M. S. Andersen. On semiseparable kernels and efficient implementation for regularized system identification and function estimation. Automatica, 132:109682, 2021
2021
-
[35]
M. Zorzi. Nonparametric identification of Kronecker networks. Automatica, 145:110518, 2022
2022
-
[36]
R. J.G.B. Campello, G. Favier, and W. C. do Amaral. Optimal expansions of discrete-time Volterra models using Laguerre functions. Automatica, 40(5):815–822, 2004
2004
-
[37]
Kekatos and G
V. Kekatos and G. B. Giannakis. Sparse Volterra and polynomial regression models: Recoverability and estimation. IEEE Transactions on Signal Processing , 59(12):5907–5920, 2011
2011
-
[38]
M. S. Andersen and T. Chen. Smoothing splines and rank structured matrices: Revisiting the spline kernel. SIAM Journal on Matrix Analysis and Applications, 41(2):389–412, 2020. 17
2020
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.