REVIEW 3 major objections 4 minor 30 references
Bayesian Modeling and Estimation of Linear Time-Varying Systems using Neural Networks and Gaussian Processes
T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Treating the impulse response as a random process lets one estimator recover an LTI filter from a single noisy pair and track a smoothly varying LTV response.
desk verdict Modest but honest: the h = mu + E decomposition is old, the posterior interpretation and single-observation deconvolution experiment are useful, and the LTV tracking claim is real but contingent on a matched smoothness prior. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the decomposition of the impulse response into a posterior mean and a zero-mean fluctuation process, $h = \mu + E$, together with the identity $\mathrm{Var}[g[n]|D] = f[n]^\top \Sigma[n] f[n]^*$ that carries uncertainty from the taps to the output. This identity is what makes predictive uncertainty depend on the probing input and what lets posterior samples of the filter be pushed forward to derived quantities like the denoised signal, the transfer function, and the cross-correlation function. For the LTV experiment the additional mechanism is a truncated Gaussian-process prior over each tap with a squared-exponential kernel in time, block-diagonal across taps, used as a KL regularizer on a Bayesian latent impulse-response layer whose parameters are produced by a deterministic CNN encoder; stitching the windowed posterior means gives the global time-varying estimate.
What would settle it
Run the LTV estimator on ground-truth responses whose three base FIRs switch abruptly within a few steps, faster than the Gaussian-process length scale of 16, and compare the stitched posterior mean against ground truth; strong tracking degradation would show the result is carried by the smoothness prior. A complementary check is to apply the single-observation estimator to a known FIR that is much longer than the 16-tap prior support and test whether the nominal 90–95 percent credible intervals remain conservative.
Extended reading notes
Core claim
The paper's central claim is that representing the impulse response as a stochastic process through $h(t,\tau) = \mu(t,\tau) + E(t,\tau)$ yields a posterior representation that unifies intrinsic channel variability and epistemic uncertainty: when inference favors time-invariant responses, a deterministic unknown LTI system induces an LTIE posterior law whose mean is time-invariant, and the same law can also describe a genuinely random channel. The output variance is then signal-dependent, since $\mathrm{Var}[g[n]|D] = f[n]^\top \Sigma[n] f[n]^*$ with posterior tap covariance $\Sigma[n]$, so uncertainty vanishes when the input is silent. The paper further shows that this representation supports practical estimators: a mean-field Bayesian convolutional layer with a zero-mean Gaussian prior recovers a 16-tap FIR from one noisy earthquake-signal pair and propagates credible intervals to denoised signals, transfer functions, and cross-correlations; a Gaussian-process prior over 32-step windows with an RBF kernel over time allows an amortized CNN encoder to track a smoothly interpolated LTV response from a single time series; and in a 15,000-pair simulated ambient-noise tomography sweep the Bayesian mean impulse response reaches a lower mean target-curve error than classical CCF stacking. Throughout, the paper states that these experiments are demonstrations of the utility of the posterior representation under misspecification, not direct confirmation that the stochastic impulse-response model is the physical data-generating mechanism.
Load-bearing premise
The load-bearing premise is that the true impulse response is smooth at the temporal scale encoded by the Gaussian-process prior (RBF kernel amplitude 2.0 and length scale 16 over 32-step windows), because single-observation local inference is otherwise severely ill-posed.
Editorial extensions
If this is right
- If the central claim is right, system identification from a single noisy input-output pair is a tractable variational inference task, with posterior intervals replacing point estimates of the filter.
- The same posterior law can be read either as intrinsic channel randomness or as epistemic uncertainty about a deterministic system, so an LTIE posterior is the natural object to report when only time-invariant behavior is assumed.
- Because output uncertainty is quadratic in the input and vanishes without excitation, predictions from the framework are self-signaling about which parts of the system are actually probed.
- The Bayesian mean impulse response reaching a lower error floor than classical CCF stacking in simulated tomography suggests the method can work without spectral whitening and can retain spectral-amplitude information.
- Smoothly varying LTV responses can be tracked from one time series if the temporal variation is slow relative to the Gaussian-process prior length scale.
Reading between the lines
- A natural next step the paper leaves untested is using the posterior spread as a stopping rule: in the ANT sweep the Bayesian error floor saturates earlier, so the posterior variance could decide how many receiver pairs are enough.
- The signal-dependent variance formula predicts a falsifiable contrast with additive-noise models: probing with an input that has a spectral null should shrink the posterior output variance in that band even when epistemic uncertainty about the corresponding taps remains, and this could be measured directly.
- The LTV experiment could be extended by learning the GP length scale or using a spectral mixture kernel; the paper lists richer priors as future work but does not test whether tracking survives when the true variation speed is unknown.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper develops a Bayesian framework for identifying linear time-varying (LTV) systems in which the impulse response h(t,τ) itself is modeled as a stochastic process and decomposed as h = μ + E, with μ the posterior mean and E zero-mean posterior fluctuations. It derives first- and second-order output statistics under this representation, defines the Linear Time-Invariant in Expectation (LTIE) class as the case where the posterior mean is time-invariant, and then presents three synthetic experiments: single-observation LTI impulse-response regression with a Bayesian convolutional layer, simulated ambient noise tomography (ANT) comparing a Bayesian mean impulse response with classical CCF stacking, and LTV tracking using an amortized variational estimator with a Gaussian-process prior. The paper is explicit throughout that the experiments are demonstrations of the posterior representation under deliberate model misspecification, not evidence that the physical data-generating mechanism is stochastic.
Significance. If the empirical claims are supported, the framework would provide a useful unified posterior representation for uncertainty-aware system identification, with practical variational estimators that propagate uncertainty from the impulse response to derived quantities such as denoised signals, transfer functions, and cross-correlation functions. The paper has clear strengths: the moment derivations in Eqs. (4), (12), and (13) are correct under the stated integrability and measurability assumptions; the single-observation experiment includes a 100-replication coverage check; the authors honestly frame the experiments as misspecification tests and distinguish epistemic from intrinsic uncertainty; and the code and data are made publicly available. However, the empirical case rests on single synthetic datasets for the two headline applications, and the LTV experiment relies on a prior whose influence in the optimized objective is very weak and whose timescale is matched to the generative model without sensitivity analysis. These issues do not undermine the theoretical framework, but they currently limit the strength of the claimed practical demonstrations.
major comments (3)
- [Sec. 4.3.2, Eq. (19); Appendix A] The LTV experiment claims that a structured GP prior regularizes an otherwise ill-posed regression, but the optimized objective places almost no weight on that prior. In Eq. (19), the loss is L = E_q ||y_window - g_hat_window||^2 + β·KL(q || GP_prior), and Appendix A reports β = 5×10^-5. With an observation-noise standard deviation of 0.2 in the generative model, a proper ELBO would have a data-weight coefficient of 1/(2σ^2) = 12.5 and a KL coefficient of 1; here the KL term is smaller than the data term by roughly five orders of magnitude. The apparent temporal smoothness may therefore come from the CNN encoder, the overlapping-window architecture, or the stitching procedure rather than from the GP prior. The paper should include an ablation or sensitivity analysis over β (e.g., β = 1 or an intermediate value) and ideally over the prior length scale, to substantiate the claim that the GP prior is doing the regularization work.
- [Sec. 4.3.1 and Appendix A] The LTV tracking demonstration lacks sensitivity analysis with respect to the temporal variation rate of the ground truth. The ground-truth response is generated by smooth interpolation among three FIRs via weights α_i[n] in Eq. (18), while the GP prior uses an RBF kernel with amplitude 2.0 and length scale 16 over 32-sample windows. The rate at which α_i[n] varies is not reported, and no experiment varies either the true variation rate or the prior length scale. As the paper acknowledges in Section 5, LTV regression quality is contingent on prior choice, but without any quantitative study of this contingency the claim that the method 'can track a continuously varying LTV impulse response' is established only for a smooth, prior-matched regime. Adding experiments with faster-varying ground truths and with mismatched prior length scales would directly address this load-bearing limitation.
- [Sec. 4.2.2, Figure C.7] The claim of a lower error floor than classical CCF stacking in the ANT experiment is based on a single simulated dataset. The reported mean target-curve errors (e.g., 0.225 versus 0.245 in the full-precision sweep, 0.241 versus 0.269 with 1-bit quantization) and minimum errors are point estimates over the pair-number sweep with no repeated-seed variability, confidence intervals, or error bars. Given that the differences are relatively small, it is plausible that they lie within seed-to-seed variation. The paper should either run multiple independent seeds/dataset realizations and report mean ± standard deviation or provide a statistical test. In addition, the 1-bit comparison changes both quantization and the Bayesian batch size (2000 to 4000), which confounds the comparison.
minor comments (4)
- [Sec. 3.2, Eq. (14)] The symbol 'B' is used to denote 'defined as' without an explicit explanation; please define it at first use or replace it with a standard notation such as ':='.
- [Appendix A, LTV experiment] The statement that 'the 291 unique windows were tiled twice for training' is unclear; please state the length of the input signal and explain how 291 follows from the window length and stride so that the setup is reproducible from the text alone.
- [Sec. 4.2.1 and Appendix C] The ANT method is described at a high level ('deconvolve the source characteristics'), but the network architecture, the exact regression target, and the loss function used for the ANT experiment are not specified as fully as in the other experiments. Expanding this description would improve reproducibility despite the availability of code.
- [Sec. 3.1, Eqs. (5)-(7)] The sentence 'A proof is omitted' for the convergence of the averaged periodogram is unnecessary, since the statement is a direct consequence of the law of large numbers, but if kept it should be integrated into the surrounding argument rather than standing alone.
Circularity Check
No significant circularity: the formal identities are definitions or direct LLN consequences, and the experimental claims are self-contained synthetic benchmarks with explicit, non-fitted prior choices.
full rationale
No circular step meeting the evidentiary bar could be identified. Section 3's development is a chain of definitional identities: h = mu + E with mu = E[h|D] (Eq. 3), the output mean and variance formulas (Eqs. 4, 12, 13), and the finite-time ensemble spectrum (Eqs. 6-7). These are presented as consequences of the stipulated probabilistic law, and the paper explicitly disclaims mathematical novelty for the decomposition itself ('The decomposition itself is not mathematically novel once a law over h is specified'). The experimental claims are genuine synthetic tests: the single-observation LTI experiment and the ANT experiment compare a variational estimator against known ground truth or a classical baseline, with hyperparameters fixed in advance and reported in Appendix A rather than fitted to the target outputs; the coverage checks are repeated-trial evaluations. The LTV experiment uses an RBF GP prior with amplitude 2.0 and length scale 16 while the ground truth is generated by smooth interpolation between three FIRs, which is a prior-truth timescale match that limits the scope of the demonstration, but the prior hyperparameters are not estimated from the ground-truth impulse response, and Section 5 explicitly concedes that LTV regression quality is contingent on prior choice. No load-bearing self-citations, imported uniqueness theorems, or ansatz-smuggling citations occur: references to Bello, WSSUS, Rician fading, and GP system identification are background contextualization, not the argument's load-bearing premises. The omitted proof of periodogram convergence is a direct pointwise law-of-large-numbers application, not a circular step. The LTIE concept is acknowledged to be a generalization of existing mean-LTI/Rician notions, so it is not a disguised renaming presented as a derivation.
Assumptions & free parameters
free parameters (8)
- BNN prior scale alpha (kernel stddev 1/sqrt(kernel size)) =
1 in single-observation and ANT experiments; implied prior stddev 1/sqrt(kernel size)
- Observation noise scale sigma_obs =
0.5 (single-observation), 0.2 (LTV), residual-estimated in ANT
- GP prior RBF amplitude =
2.0
- GP prior RBF length scale =
16
- GP kernel jitter =
1e-5
- KL scaling beta =
5e-5 in LTV; 1/(number of distinct pairs) in single-observation and ANT
- Local window length and stride =
length 32, unit stride, tiled twice in LTV experiment
- ANT simulation source parameters =
10,000 decaying sinusoidal pulses, amplitude [-1.5,1.5], frequency [0.1,10] Hz, decay tau [0.5,2.5]
assumptions (7)
- standard math Fubini/Tonelli conditions for exchanging expectation and integral
- domain assumption Integrability and finite second moments of h and E
- domain assumption Persistency of excitation of the input
- domain assumption Cross-correlation approximates the Green's function in ANT simulation
- domain assumption Local stationarity and negligible within-window non-stationarity
- ad hoc to paper Gaussian observation noise and mean-field variational family
- ad hoc to paper Smooth temporal evolution of the LTV impulse response
invented entities (1)
-
Linear Time-Invariant in Expectation (LTIE) system class
Cite this review
Pith. "Pith review of Bayesian Modeling and Estimation of Linear Time-Varying Systems using Neural Networks and Gaussian Processes." pith.science (2026). https://pith.science/paper/FL3UB5VH
@misc{pith2026250712878,
author = {Pith},
title = {Pith review of: Bayesian Modeling and Estimation of Linear Time-Varying Systems using Neural Networks and Gaussian Processes},
year = {2026},
howpublished = {\url{https://pith.science/paper/FL3UB5VH}},
note = {Machine review of arXiv:2507.12878}
}
abstract
The identification of Linear Time-Varying (LTV) systems from input-output data is a fundamental yet challenging ill-posed inverse problem. This work introduces a unified Bayesian framework that models the system's impulse response, $h(t, \tau)$, as a stochastic process. We decompose the response into a posterior mean and a random fluctuation term, a formulation that provides a principled approach for quantifying uncertainty, unifies intrinsic channel variability and epistemic uncertainty through a common posterior representation, and naturally defines a new, useful system class we term Linear Time-Invariant in Expectation (LTIE). To perform inference, we leverage modern machine learning techniques, including Bayesian neural networks and Gaussian Processes, using scalable variational inference. We demonstrate through a series of experiments that our framework can infer the properties of an LTI system from a single noisy input-output pair, including under deliberate additive-noise misspecification, achieve a lower overall error floor than the classical CCF stacking baseline in a simulated ambient noise tomography setting, and track a continuously varying LTV impulse response by using a structured Gaussian Process prior. This work provides a flexible and robust methodology for uncertainty-aware system identification in dynamic environments.
Reference graph
Works this paper leans on
-
[10]
arXiv preprint arXiv:1711.10604 URL:https://arxiv.org/abs/1711
Tensorflow distributions. arXiv preprint arXiv:1711.10604 URL:https://arxiv.org/abs/1711. 10604,arXiv:1711.10604. Ekstr¨om, G., Abers, G.A., Webb, S.C.,
-
[13]
IEEE Computational Intelligence Magazine 17, 29–48
Hands-on bayesian neural networks—a tutorial for deep learning users. IEEE Computational Intelligence Magazine 17, 29–48. doi:10.1109/MCI.2022.3155327. Kailath, T.,
arXiv 2022
-
[17]
arXiv preprint arXiv:2010.14664 doi:10.48550/arXiv.2010.14664,arXiv:2010.14664
System identification via meta-learning in linear time-varying environments. arXiv preprint arXiv:2010.14664 doi:10.48550/arXiv.2010.14664,arXiv:2010.14664. Loshchilov, I., Hutter, F.,
-
[19]
Nonlinear System Identification: From Classical Approaches to Neural Networks, Fuzzy Models, and Gaussian Processes. 2nd ed., Springer. doi:10.1007/978-3-030-47439-3. Pillonetto, G., Ljung, L.,
-
[24]
Electric Power Systems Research 235, 110860
Bayesian physics-informed neural net- works for system identification of inverter-dominated power systems. Electric Power Systems Research 235, 110860. URL:https://www.sciencedirect.com/science/article/pii/S0378779624007466, doi:10. 1016/j.epsr.2024.110860. Susik, M., Sbalzarini, I.F.,
-
[25]
Forensic Science 17 International: Genetics 65, 102890
Variational inference accelerates accurate dna mixture deconvolution. Forensic Science 17 International: Genetics 65, 102890. doi:10.1016/j.fsigen.2023.102890. epub 2023 May
-
[26]
TensorFlow Probability API documentation, accessed 2026-03-21
tfp.layers.convolution1dflipout.https://www.tensorflow.org/probability/api_ docs/python/tfp/layers/Convolution1DFlipout. TensorFlow Probability API documentation, accessed 2026-03-21. Wen, Y ., Vicol, P., Ba, J., Tran, D., Grosse, R.,
work page 2026
-
[27]
Systems & Control Letters 54, 325–329
A note on persistency of excitation. Systems & Control Letters 54, 325–329. URL:https://eprints.soton.ac.uk/262195/1/PersistencyExcitation. pdf, doi:10.1016/j.sysconle.2004.09.003. Yoo, D.S., Stark, W.E.,
Show all 30 references
-
[28]
IEEE Transac- tions on Wireless Communications 4, 1575–1584
Characterization of wssus channels: Normalized mean square covariance. IEEE Transac- tions on Wireless Communications 4, 1575–1584. doi:10.1109/TWC.2004.843046. Zhang, G., Kingsbury, N.,
2004
-
[29]
Fast l0-based image deconvolution with variational bayesian inference and majorization- minimization, in: Proceedings of the 2013 IEEE Global Conference on Signal and Information Processing (Glob- alSIP), IEEE. pp. 1081–1084. URL:https://www-sigproc.eng.cam.ac.uk/foswiki/pub/M...
2013
-
[30]
Earth- quakes
Sparse bayesian deep learning for dynamic system identification. Automatica 144, 110489. URL:https://doi.org/10.1016/j.automatica.2022.110489, doi:10.1016/j. automatica.2022.110489. 18 Appendix A. Experimental Settings For reproducibility, we summarize here the main settings u...
2022
-
[1957]
Bulletin of the Earthquake Research Institute, University of Tokyo 35, 415–456
Space and time spectra of stationary stochastic waves, with special reference to microtremors. Bulletin of the Earthquake Research Institute, University of Tokyo 35, 415–456. URL:https://doi.org/10.15083/ 0000033938, doi:10.15083/0000033938. Aubel, C., B ¨olcskei, H.,
-
[1962]
IRE Transactions on Information Theory 8, 229–236
Measurements on time-variant communication channels. IRE Transactions on Information Theory 8, 229–236. doi:10.1109/TIT.1962.1057748. Kingma, D.P., Welling, M.,
1962
-
[1963]
IRE Transactions on Communications Systems 11, 360–393
Characterization of randomly time-variant linear channels. IRE Transactions on Communications Systems 11, 360–393. doi:10.1109/TCOM.1963.1088793. Bensen, G.D., Ritzwoller, M.H., Barmin, M.P., Levshin, A.L., Lin, F., Moschetti, M.P., Shapiro, N.M., Yang, Y .,
1963
-
[1965]
Journal of the Royal Statistical Society: Series B (Methodological) 27, 204–237
Evolutionary spectra and non-stationary processes. Journal of the Royal Statistical Society: Series B (Methodological) 27, 204–237. URL:https://doi.org/10.1111/j.2517-6161.1965.tb01488.x, doi:10.1111/j.2517-6161.1965.tb01488.x. Rezende, D.J., Mohamed, S., Wierstra, D.,
1965
-
[1990]
IEEE Transactions on Neural Networks 1, 4–27
Identification and control of dynamical systems using neural networks. IEEE Transactions on Neural Networks 1, 4–27. doi:10.1109/72.80202. Nelles, O.,
-
[1997]
The Annals of Statistics 25, 1–37
Fitting time series models to nonstationary processes. The Annals of Statistics 25, 1–37. URL: https://doi.org/10.1214/AOS/1034276620, doi:10.1214/AOS/1034276620. Damianou, A., Lawrence, N.D.,
-
[2004]
Physical Review E 69, 046610
Extracting the green’s function from the correlation of coda waves: A derivation based on stationary phase. Physical Review E 69, 046610. URL:https://doi.org/10.1103/PhysRevE.69.046610, doi:10.1103/ PhysRevE.69.046610. Stock, S., Babazadeh, D., Becker, C., Chatzivasileiadis, S.,
-
[2005]
Springer, pp
Image deconvolution, in: Microscopy Techniques. Springer, pp. 355–378. URL:https: //person.dibris.unige.it/bertero-mario/papers-PDF/ASI.pdf, doi:10.1007/1-4020-3616-7_17. Bhouri, M.A., Perdikaris, P.,
-
[2006]
Bulletin of the Seismological Society of America 96, 1182–1191
Retrieval of the green’s function from cross correlation: The canonical elastic problem. Bulletin of the Seismological Society of America 96, 1182–1191. URL: https://doi.org/10.1785/0120050181, doi:10.1785/0120050181. Sendera, M., Sorkhei, A., Ku ´smierczyk, T.,
-
[2007]
Geophysical Journal International 169, 1239–1260
Processing seismic ambient noise data to obtain reliable broad-band surface wave dispersion measurements. Geophysical Journal International 169, 1239–1260. URL:https://academic.oup.com/gji/article/169/3/ 1239/626431, doi:10.1111/j.1365-246X.2007.03374.x. Bertero, M., Boccacci, P.,
-
[2009]
Geophysical Research Letters 36, L18301
Determination of surface-wave phase velocities across USArray from noise and Aki’s spectral formulation. Geophysical Research Letters 36, L18301. URL:https://doi.org/10. 1029/2009GL039131, doi:10.1029/2009GL039131. Hallemans, N., Lataire, J., Pintelon, R.,
-
[2013]
CoRR abs/1312.6114
Auto-encoding variational bayes. CoRR abs/1312.6114. URL:https://arxiv. org/abs/1312.6114. Kozachek, O., Nikolaev, N., Slita, O., Bobtsov, A.,
-
[2015]
2568–2572
Density criteria for the identification of linear time-varying systems, in: 2015 IEEE International Symposium on Information Theory (ISIT), pp. 2568–2572. doi:10.1109/ISIT.2015.7282920. Bello, P.A.,
2015
-
[2017]
Bayesian Identification of Linear Dynamic Systems: Synthesis of Kernels in the LTI Case and Beyond. Ph.D. thesis. Technische Universiteit Eindhoven. Eindhoven, The Netherlands. URL:https://pure.tue. nl/ws/files/77531824/20171010_Darwish.pdf. proefschrift. Dau, H.A., Bagnall, A...
-
[2019]
IEEE/CAA Journal of Automatica Sinica 6, 1293–1305
The UCR time series archive. IEEE/CAA Journal of Automatica Sinica 6, 1293–1305. URL:https://doi.org/ 10.1109/JAS.2019.1911747, doi:10.1109/JAS.2019.1911747. used for the Earthquakes dataset from the UCR Archive. Dillon, J.V ., Langmore, I., Tran, D., Brevdo, E., Vasudevan, S....
2019
-
[2020]
IFAC-PapersOnLine 53, 1001–1006
Nonparametric identification of linear time-varying systems using gaussian process regression. IFAC-PapersOnLine 53, 1001–1006. URL:https://doi.org/10.1016/j.ifacol.2020. 12.1277, doi:10.1016/j.ifacol.2020.12.1277. Jospin, L.V ., Laga, H., Boussaid, F., Buntine, W., Bennamoun, M.,
2020 doi
-
[2022]
Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 380, 20210201
Gaussian processes meet neuralodes: A bayesian framework for learning the dy- namics of partially observed systems from scarce and noisy data. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 380, 20210201. URL:https://doi.org/...
-
[2023]
Proceedings of the National Academy of Sciences of the United States of America 120, e2218197120
Full bayesian identification of linear dynamic systems using stable kernels. Proceedings of the National Academy of Sciences of the United States of America 120, e2218197120. URL:https://www. pnas.org/doi/10.1073/pnas.2218197120, doi:10.1073/pnas.2218197120. Priestley, M.B.,
-
[2024]
Parameter Identification Algorithm for a LTV System with Partially Unknown State Matrix. pp. 306–319. doi:10.1007/978-3-031-71360-6_23. Lin, S., Wang, H., Zhang, J.,
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.