Neural networks for nonlinear regression with serially correlated disturbances: Evidence from cloud cover

Sebastian Jensen; Siem Jan Koopman

REVIEW 2 major objections 2 minor 15 references

Neural networks for nonlinear regression with serially correlated disturbances: Evidence from cloud cover

T0 review · 2 major / 2 minor · reviewed 2026-06-26 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Embedding ARMA error structures into feedforward neural networks yields more accurate cloud cover predictions than LSTM networks or linear models with serially correlated errors.

desk verdict The paper embeds ARMA error structures into feedforward nets as an alternative to lagged inputs for nonlinear time-series regression, with simulation recovery and robustness plus better Mediterranean cloud-cover forecasts than LSTM, but the real-data edge depends on the chosen orders matching an unknown DGP. read the letter →

arxiv 2606.22483 v1 pith:22UB2RD5 submitted 2026-06-21 econ.EM stat.AP

Sebastian Jensen , Siem Jan Koopman This is my paper · ORCID

classification econ.EMstat.AP

keywords neuralnetworksARMAstructuresserialcorrelationnonlinearregressioncloudcoverpredictiontimeseriesforecastingMediterraneanfeedforward

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces a neural network model that directly incorporates autoregressive moving average structures to handle serial correlation in the disturbances of nonlinear regressions. This serves as an alternative to using lagged dependent variables for capturing temporal dependence. Simulations across varying time series lengths and noise levels show that the model recovers both the underlying regression function and the error process reliably. In an empirical application to cloud cover data over a grid of locations in and around the Mediterranean Sea, the approach produces lower prediction errors than competing methods, with the largest improvements appearing in mountainous terrain where nonlinear effects are stronger.

What carries the argument

The feedforward neural network with embedded ARMA error dynamics, which separates the nonlinear regression function from an explicit parametric model of the serially correlated disturbances.

What would settle it

A replication study on an independent grid of cloud cover observations or another environmental time series showing that the proposed model's mean squared prediction error is not lower than that of an LSTM or a linear ARMA model would falsify the superiority claim.

Watch

Extended reading notes

Core claim

The central claim is that a feedforward neural network augmented with an ARMA specification for the disturbances models nonlinear mean functions and serial correlation in errors more effectively than either lagged-variable approaches or standard recurrent architectures, as evidenced by accurate recovery in simulations and superior out-of-sample forecasts for Mediterranean cloud cover, particularly in regions with pronounced nonlinearity.

Load-bearing premise

The selected ARMA orders and network architecture are sufficient to represent the true serial correlation and nonlinearity present in the data-generating process.

Editorial extensions

If this is right

The model recovers regression functions of varying complexity and the underlying ARMA error dynamics across a range of sample sizes and signal-to-noise ratios.
Predictive performance remains robust when lagged variables are omitted or the error dynamics are misspecified.
Out-of-sample accuracy exceeds that of linear models with serially correlated errors and long short-term memory networks, with gains concentrated in mountain areas.
The separation of nonlinear mean modeling from explicit error dynamics provides an alternative route to handling temporal dependence in regression settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same structure could be tested on other gridded environmental or economic series that exhibit both nonlinearity and autocorrelation to identify domains where the separation of mean and error components matters most.
If the ARMA component can be estimated jointly with the network weights, the approach may offer a route to more interpretable neural network residuals in time series applications.
Extending the framework to higher-order or seasonal ARMA specifications might further improve performance on data with periodic patterns, such as diurnal or annual cycles in cloud cover.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

The key point is that this paper embeds ARMA error structures directly into a feedforward neural network for nonlinear time-series regression, offering an alternative to including lagged dependent variables. It shows good recovery in simulations and better predictions for Mediterranean cloud cover than LSTM or linear models.

The new part is the integration of ARMA into the network rather than treating serial correlation through lags or recurrent architectures. The simulations demonstrate that the method recovers the underlying functions and error processes even under misspecification like wrong ARMA orders or omitted lags. The empirical section applies it to cloud cover data and reports improved accuracy, particularly in mountainous areas.

That said, the empirical superiority rests on the assumption that the selected ARMA orders and network architecture match the unknown data-generating process in the cloud cover data. The simulations use known DGPs, so they don't fully address whether the gains in the real application are robust or could be artifacts of the specification. The paper is limited to one environmental variable and region, which narrows the scope.

This work is for econometricians and climate researchers dealing with nonlinear relationships in time series with serial correlation. It has enough simulation and empirical content to merit peer review, though the full manuscript would need to provide more details on model selection and robustness checks to strengthen the claims.

Referee Report

2 major / 2 minor

Summary. The paper proposes a feedforward neural network that embeds ARMA error dynamics to model nonlinear regression with serially correlated disturbances, offering an alternative to explicit lagged-variable specifications. Simulations across varying time-series lengths and signal-to-noise ratios demonstrate accurate recovery of both the regression function and the error process, together with robustness to omitted lags and incorrect ARMA orders. In an empirical application to gridded cloud-cover data over the Mediterranean, the NN-ARMA model produces lower out-of-sample prediction errors than linear ARMA models and LSTM networks, with the largest gains reported in mountainous sub-regions.

Significance. If the reported predictive gains survive further scrutiny, the approach supplies a practical, parsimonious way to combine flexible nonlinearity with explicit serial-correlation modeling in econometric time-series applications. The cloud-cover exercise is a relevant test case for environmental econometrics and climate-impact studies, where both nonlinearity and temporal dependence are common. The simulation evidence of robustness to misspecification is a positive feature, though its value hinges on how closely the tested DGPs resemble the unknown cloud-cover process.

major comments (2)

[Empirical application] Empirical section (cloud-cover application): the headline claim that NN-ARMA outperforms LSTM and linear ARMA rests on the adequacy of the chosen ARMA(p,q) orders and network depth/width for the unknown Mediterranean cloud-cover DGP. The manuscript should report (i) the exact procedure used to select p and q, (ii) the grid of architectures examined, and (iii) sensitivity checks that vary these choices; without such evidence the out-of-sample gains could be artifacts of specification rather than a general methodological improvement.
[Simulation study] Simulation section: while robustness to omitted lags and wrong ARMA orders is asserted, the paper must clarify whether the Monte Carlo DGPs include features known to be present in cloud-cover series (long memory, regime shifts, or spatial-temporal interactions). If these features are absent from the simulation design, the reported robustness does not directly address the skeptic's concern about the real-data DGP.

minor comments (2)

[Abstract] The abstract states that improvements are 'particularly pronounced in mountain areas' but does not report the quantitative difference in RMSE or MAE between mountain and non-mountain grids; adding these numbers would strengthen the claim.
[Model section] Notation for the embedded ARMA component should be made fully explicit (e.g., how the MA terms enter the network output layer) to facilitate replication.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We address each major point below and indicate where revisions will be made.

read point-by-point responses

Referee: [Empirical application] Empirical section (cloud-cover application): the headline claim that NN-ARMA outperforms LSTM and linear ARMA rests on the adequacy of the chosen ARMA(p,q) orders and network depth/width for the unknown Mediterranean cloud-cover DGP. The manuscript should report (i) the exact procedure used to select p and q, (ii) the grid of architectures examined, and (iii) sensitivity checks that vary these choices; without such evidence the out-of-sample gains could be artifacts of specification rather than a general methodological improvement.

Authors: We agree that these details are necessary to substantiate the empirical claims. The revised manuscript will add a dedicated subsection reporting (i) the ARMA order selection procedure, (ii) the full grid of network depths and widths that were examined, and (iii) sensitivity results across alternative choices. These additions will clarify that the reported gains are not driven by a single specification. revision: yes
Referee: [Simulation study] Simulation section: while robustness to omitted lags and wrong ARMA orders is asserted, the paper must clarify whether the Monte Carlo DGPs include features known to be present in cloud-cover series (long memory, regime shifts, or spatial-temporal interactions). If these features are absent from the simulation design, the reported robustness does not directly address the skeptic's concern about the real-data DGP.

Authors: The Monte Carlo design examines robustness specifically to omitted lags and incorrect ARMA orders under stationary processes. It does not incorporate long memory, regime shifts, or spatial-temporal interactions. We will revise the simulation section to state this scope explicitly and to note that the empirical application provides the relevant evidence for the more complex cloud-cover process. This distinction will be made clear without overstating the simulation results. revision: partial

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: model definition and empirical validation are independent of fitted inputs

full rationale

The paper defines a feedforward NN with embedded ARMA error structure as an explicit alternative to lagged-variable approaches. Simulations recover known DGPs under controlled misspecification; empirical results are out-of-sample predictive comparisons on Mediterranean cloud-cover grids against LSTM and linear ARMA baselines. No equations reduce to their own fitted parameters by construction, no self-citation chain supports a uniqueness claim, and no ansatz is smuggled via prior work. The central claim (superior predictive accuracy) rests on direct performance metrics rather than tautological re-labeling of inputs.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

Abstract-only review provides no equations, so no free parameters, axioms, or invented entities can be identified.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Neural networks for nonlinear regression with serially correlated disturbances: Evidence from cloud cover." pith.science (2026). https://pith.science/paper/22UB2RD5

@misc{pith2026260622483,
  author       = {Pith},
  title        = {Pith review of: Neural networks for nonlinear regression with serially correlated disturbances: Evidence from cloud cover},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/22UB2RD5}},
  note         = {Machine review of arXiv:2606.22483}
}

read the original abstract

We propose a new treatment of nonlinear regression with serially correlated disturbances that incorporates autoregressive moving average structures into feedforward neural networks. The resulting model provides an alternative to modeling temporal dependence using lagged variables. In simulations, the proposed method accurately recovers regression functions of varying complexity and the underlying error dynamics across a range of time-series lengths and signal-to-noise ratios. Finite-sample properties and out-of-sample predictive performances are shown to be robust to model misspecification induced by omitted lagged variables and incorrect specification of the error dynamics. Cloud cover is an important factor in climate projections. In an empirical study of cloud cover prediction for a grid of locations within and around the Mediterranean Sea, our proposed model yields more accurate predictions than existing methods, including long short-term memory networks. Improvements are observed broadly and are particularly pronounced in mountain areas relative to linear models with serially correlated errors, consistent with the presence of stronger nonlinear effects in cloud composure in such regions.

Figures

Figures reproduced from arXiv: 2606.22483 by the authors.

**Figure 2.** Sampling distribution of the ARMA coefficient estimates for the hump-shaped (solid) [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗

**Figure 3.** Out-of-sample results for NN and NNARMA (correctly specified) models with Box plots [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: Out-of-sample results for NNARMA misspecified models (NNARMA(1 [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

**Figure 5.** Figure 5: Out-of-sample results for a selection of NN models (NN(2 [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

**Figure 6.** Figure 6: Topography map. Figure 7 shows time series plots for the example locations labeled 1–5. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

**Figure 7.** Figure 7: NNARMA predictions over the tests sample (red) together with observed cloud fractional [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

**Figure 8.** Figure 8: Empirical distribution of ARMA specifications across the 861 geographical locations [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]

**Figure 9.** Figure 9: Predictions over the tests sample (red) together observed cloud fractional cover (black) [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]

**Figure 10.** Figure 10: Relative MSEs over the test sample normalized to NNARMA (left), where values above [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]

**Figure 11.** Figure 11: Relative MSEs over the test sample normalized to NNARMA (left), where values above [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 2 canonical work pages

[1]

Altman, N. S. (1990). Kernel smoothing of data with correlated errors.Journal of the American Statistical Association 85(411), 749–759. Athey, S. and G. W. Imbens (2019). Machine learning methods that economists should know about.Annual Review of Economics 11(Volume 11, 2019), 685–725. Bach, F. (2017). Breaking the curse of dimensionality with convex neur...

1990
[2]

Bengtsson, L. (2010). The global atmospheric water cycle.Environmental Research Letters 5(2), 025202. Bennedsen, M., E. Hillebrand, and S. Jensen (2023). A neural network approach to the environ- mental Kuznets curve.Energy Economics 126, 106985. Bony, S., B. Stevens, D. Frierson, C. Jakob, M. Kageyama, R. Pincus, T. Shepherd, S. Sherwood, A. Siebesma, M....

work page doi:10.17605/osf.io/kqdgx 2010
[3]

Jones, R. H. (1980). Maximum likelihood fitting of arma models to time series with missing observations.Technometrics 22, 389–395. Jozefowicz, R., W. Zaremba, and I. Sutskever (2015). An empirical exploration of recurrent network architectures. InProceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, ...

work page Pith review arXiv 1980
[4]

26 Levinson, N. (1946). The wiener (root mean square) error criterion in filter design and prediction. Journal of Mathematics and Physics 25(1-4), 261–278. Li, Q. and J. Racine (2007).Nonparametric Econometrics: Theory and Practice. Princeton, NJ: Princeton University Press. Liu, H., I. Koren, O. Altaratz, and M. D. Chekroun (2023). Opposing trends of clo...

1946
[5]

Rumelhart, D. E., G. E. Hinton, and R. J. Williams (1986). Learning representations by back- propagating errors.Nature 323, 533–536. Schmetz, J., P. Pili, S. Tjemkes, D. Just, J. Kerkmann, S. Rota, and A. Ratier (2002). An introduction to meteosat second generation (msg).Bulletin of the American Meteorological Society 83(7), 977 –

1986
[6]

Schmidt-Hieber, J. (2020). Nonparametric regression using deep neural networks with ReLU acti- vation function.The Annals of Statistics 48(4), 1875 –

2020
[7]

Silverman, B. W. (1986).Density estimation for statistics and data analysis. London: Chapman and Hall. Slingo, J. M. (1987). The development and verification of a cloud prediction scheme for the ecmwf model.Quarterly Journal of the Royal Meteorological Society 113(477), 899–927. Smith, R. B. (1979). The influence of mountains on the atmosphere. Volume 21 ...

1986
[8]

Stephens, G. L., J. Li, M. Wild, C. A. Clayson, N. Loeb, S. Kato, T. L’Ecuyer, P. W. Stackhouse, M. Lebsock, and T. Andrews (2012). An update on earth’s energy balance in light of the latest global observations.Nature Geoscience 5(10), 691–696. Stevens, B. and S. Bony (2013). What are climate models missing?Science 340(6136), 1053–1054. Stone, C. J. (1982...

2012

Show all 15 references

[9]

Hicks, M

Svennevik, H., S. Hicks, M. Riegler, T. Storelvmo, and H. Hammer (2024). A dataset for predicting cloud cover over europe.Scientific Data 11(1),

2024
[10]

Medeiros, and G

Ter¨ asvirta, T., M. Medeiros, and G. Rech (2006). Building neural network models for time series: a statistical approach.Journal of Forecasting 25(1), 49–75. 28 Vo, T. T., L. Hu, L. Xue, and S. Chen (2025). Trends in cloud covers across conus (1980–2020). Journal of Climate 3...

2006
[11]

Watson, G. S. (1964). Smooth regression analysis.Sankhy¯ a: The Indian Journal of Statistics, Series A (1961-2002) 26(4), 359–372. White, H. (1989). Some Asymptotic Results for Learning in Single-Hidden-Layer Feedforward Networks.Neural Networks 2, 425–431. White, H. and I. Do...

1964
[12]

Zelinka, M. D., T. A. Myers, D. T. McCoy, S. Po-Chedley, P. M. Caldwell, P. Ceppi, S. A. Klein, and K. E. Taylor (2020). Causes of higher climate sensitivity in cmip6 models.Geophysical Research Letters 47(1), e2019GL085782. Zhang, G. (2003). Time series forecasting using a hy...

2020
[13]

stability condition

Thus,Ccan be obtained without assumptions on σ2. The coefficientsϕ tt equal the partial autocorrelations of the processu t at lagt; see Appendix B. 31 B Jones’ reparametrization We use the reparametrization technique of Jones (1980) to transform the constrained minimization pr...

1980
[14]

At lowr, the sinusoidal function is estimated fairly well provided the sample size is large enough

and neural network applications (Bennedsen et al., 2023). At lowr, the sinusoidal function is estimated fairly well provided the sample size is large enough. It generally requires a larger sample size than for the hump-shaped function. Corresponding to the surface estimates in...

2023
[15]

grey”, “blue

Our strategy for selectingpandqis based on the BIC. We also considered the AIC, but the dis- tribution of selected ARMA specifications was less concentrated and did not converge to the true specification asTincreased. Neural network architecture Figure C.17 shows the validatio...

1993

Pith tools

Reviewed June 26, 2026 · model on record in the stance chip above.

[1] [1]

Altman, N. S. (1990). Kernel smoothing of data with correlated errors.Journal of the American Statistical Association 85(411), 749–759. Athey, S. and G. W. Imbens (2019). Machine learning methods that economists should know about.Annual Review of Economics 11(Volume 11, 2019), 685–725. Bach, F. (2017). Breaking the curse of dimensionality with convex neur...

1990

[2] [2]

Bengtsson, L. (2010). The global atmospheric water cycle.Environmental Research Letters 5(2), 025202. Bennedsen, M., E. Hillebrand, and S. Jensen (2023). A neural network approach to the environ- mental Kuznets curve.Energy Economics 126, 106985. Bony, S., B. Stevens, D. Frierson, C. Jakob, M. Kageyama, R. Pincus, T. Shepherd, S. Sherwood, A. Siebesma, M....

work page doi:10.17605/osf.io/kqdgx 2010

[3] [3]

Jones, R. H. (1980). Maximum likelihood fitting of arma models to time series with missing observations.Technometrics 22, 389–395. Jozefowicz, R., W. Zaremba, and I. Sutskever (2015). An empirical exploration of recurrent network architectures. InProceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, ...

work page Pith review arXiv 1980

[4] [4]

26 Levinson, N. (1946). The wiener (root mean square) error criterion in filter design and prediction. Journal of Mathematics and Physics 25(1-4), 261–278. Li, Q. and J. Racine (2007).Nonparametric Econometrics: Theory and Practice. Princeton, NJ: Princeton University Press. Liu, H., I. Koren, O. Altaratz, and M. D. Chekroun (2023). Opposing trends of clo...

1946

[5] [5]

Rumelhart, D. E., G. E. Hinton, and R. J. Williams (1986). Learning representations by back- propagating errors.Nature 323, 533–536. Schmetz, J., P. Pili, S. Tjemkes, D. Just, J. Kerkmann, S. Rota, and A. Ratier (2002). An introduction to meteosat second generation (msg).Bulletin of the American Meteorological Society 83(7), 977 –

1986

[6] [6]

Schmidt-Hieber, J. (2020). Nonparametric regression using deep neural networks with ReLU acti- vation function.The Annals of Statistics 48(4), 1875 –

2020

[7] [7]

Silverman, B. W. (1986).Density estimation for statistics and data analysis. London: Chapman and Hall. Slingo, J. M. (1987). The development and verification of a cloud prediction scheme for the ecmwf model.Quarterly Journal of the Royal Meteorological Society 113(477), 899–927. Smith, R. B. (1979). The influence of mountains on the atmosphere. Volume 21 ...

1986

[8] [8]

Stephens, G. L., J. Li, M. Wild, C. A. Clayson, N. Loeb, S. Kato, T. L’Ecuyer, P. W. Stackhouse, M. Lebsock, and T. Andrews (2012). An update on earth’s energy balance in light of the latest global observations.Nature Geoscience 5(10), 691–696. Stevens, B. and S. Bony (2013). What are climate models missing?Science 340(6136), 1053–1054. Stone, C. J. (1982...

2012