Pith. sign in

REVIEW 4 major objections 7 minor 23 references

Forecasting Probability Distributions of Financial Returns with Deep Neural Networks

T0 review · 4 major / 7 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Deep learning matches GARCH for value-at-risk on six major equity indexes, with LSTM and skewed Student-t leading.

desk verdict A competent, reproducible application of known probabilistic forecasting methods to equity indices, but the calibration evidence in its own Table 2 contradicts the abstract's claim of 'accurate distributional forecasts.' read the letter →

arxiv 2508.18921 v2 pith:JUSHOIEG submitted 2025-08-26 q-fin.RM cs.LG

Jakub Michańków This is my paper
classification q-fin.RMcs.LG
keywords probabilisticforecastingdeeplearningLSTM1DCNNskewedStudent'stValue-at-RiskGARCHcomparisondistributionalcalibration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that deep neural networks can forecast the full probability distribution of daily financial returns, not just point predictions. Using custom negative log-likelihood losses, 1D CNN and LSTM models directly output the parameters of Normal, Student's t, and skewed Student's t distributions. Tested on six global equity indices over more than two decades, the models produce accurate distributional forecasts and VaR exceedance rates close to theoretical levels, competitive with classical GARCH specifications. The LSTM with a skewed Student's t distribution performs best overall, suggesting that combining sequential memory with heavy-tailed, asymmetric distributions is a viable route for risk management.

What carries the argument

The engine is a conditional density network: an LSTM or 1D CNN reads the last 10 daily returns and outputs the parameters—location, scale, degrees of freedom, and skewness—of a predictive distribution. Training minimizes the negative log-likelihood of the observed return under that distribution. For the skewed Student's t case, a piecewise skewing transformation turns a symmetric t density into an asymmetric one, so the network learns heavy tails and asymmetry directly from data. This parameter-output design lets VaR and Expected Shortfall be read off analytically from the predicted quantiles.

What would settle it

Re-run the walk-forward experiment on one index, say the S&P 500, with the same expanding-window protocol, fit a standard GARCH-t by maximum likelihood, and compare its 5% and 1% VaR exceedance counts and Kupiec test p-values with the values reported in Table 5; if a correctly implemented GARCH baseline deviates materially from the reported 'best GARCH' exceedance rates, the paper's headline competitiveness claim would not hold.

Watch

Extended reading notes

Core claim

The central claim is that a neural network trained to minimize the negative log-likelihood of a parametric return distribution can serve as a practical distributional forecaster for financial risk. On daily returns of the S&P 500, BOVESPA, DAX, WIG, Nikkei 225, and KOSPI, the best configuration—an LSTM outputting skewed Student's t parameters—achieves the lowest Log Predictive Score and CRPS, better PIT calibration than Normal or symmetric t variants, and 5%/1% VaR exceedance rates (e.g., 4.50%/0.84% on the S&P 500) that satisfy Kupiec and Christoffersen tests. Against GARCH benchmarks, the best neural network exceedance rates are comparable or closer to the nominal rate, supporting the pape

Load-bearing premise

The reported competitiveness with GARCH depends on the GARCH baselines being correctly specified, estimated on the same expanding windows, and directly comparable to the neural network exceedance rates, but the paper gives no estimation details or diagnostics for those baselines.

Editorial extensions

If this is right

  • Risk managers can use LSTM-SSTD predictive distributions to compute VaR and Expected Shortfall directly from the forecasted parameters, without separate volatility modeling.
  • The skewed Student's t specification improves calibration over Normal and symmetric t, so the extra tail and asymmetry parameters carry real forecasting information rather than overfitting noise.
  • Neural distributional forecasts are competitive enough to serve as a benchmark or replacement for GARCH-based VaR systems across diverse equity markets.
  • The CNN's near-competitive performance with lower computational cost points to a practical trade-off for high-frequency risk applications.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not separate calm from crisis periods in its evaluation; testing the same LSTM-SSTD recipe on stress subsets (e.g., 2008, 2020) would reveal whether the learned tail asymmetry is stable or regime-dependent.
  • Since LSTM consistently edges out CNN on daily data, sequence memory seems to matter more than local feature extraction at this frequency; intraday data, where local patterns dominate, is a natural test of whether CNN closes the gap.
  • A direct ablation varying only the distribution family while holding architecture fixed would isolate how much of the calibration gain comes from the skewed Student-t shape rather than the network structure.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes 1D CNN and LSTM architectures for forecasting the full probability distribution of daily financial returns, using negative log-likelihood losses for Normal, Student's t, and skewed Student's t distributions. The models are evaluated with expanding-window walk-forward validation on six equity indices, with LPS, CRPS, and PIT as distributional metrics and VaR/ES backtests as risk-management metrics. The paper claims that deep learning models provide accurate distributional forecasts, that LSTM with skewed Student's t performs best, and that the models are competitive with classical GARCH models for VaR estimation.

Significance. If the claims were validated, the paper would provide a useful benchmark for deep-learning-based distributional forecasting in financial risk management and would extend the probabilistic deep-learning literature to daily equity returns. The strengths are the use of proper scoring rules, a walk-forward evaluation design, multiple asset classes, and the availability of source code. However, the paper's own calibration evidence undermines the central claim of 'accurate distributional forecasts': 17 of 18 reported PIT tests reject uniformity at the 5% level, including those for the preferred LSTM-SSTD model on two of three reported indices. In addition, the claim of superior performance rests on selecting the best model from the same test set without multiple-comparison correction, and the GARCH comparison is reported without estimation details or statistical tests. The empirical framework is reasonable and the code availability is a plus, but the current mismatch between the abstract/conclusion and the reported calibration statistics is substantial.

major comments (4)
  1. [Section 5.1, Table 2] The PIT calibration results are presented as evidence supporting the abstract's claim of 'accurate distributional forecasts,' but the table reports rejection of uniformity for 17 of 18 model-index combinations at the 5% level. The headline LSTM-SSTD model has PIT p-values of 0.0309 (S&P 500), 2.41e-07 (Nikkei 225), and 5.08e-06 (KOSPI). Only CNN-SSTD for S&P 500 (p=0.1144) does not reject. Moreover, Table 2 reports only three of the six indices, while the abstract claims evaluation across six global markets; DAX, WIG, and BOVESPA are absent from the distributional evaluation, despite Figure 2 showing DAX PIT histograms without p-values. The claim of well-calibrated distributional forecasts is therefore contradicted by the paper's own reported evidence. The authors should either weaken the claim to 'VaR exceedance rates are competitive' or provide a calibration analysis with effect sizes,
  2. [Sections 5.1-5.2, Tables 2-5] The 'best' model is selected from the same test data on which its performance is reported, and no multiple-comparison correction is applied. Table 2 identifies LSTM-SSTD as best by lowest LPS/CRPS, but no significance tests (e.g., Diebold-Mariano or a test for equal predictive ability) are provided for the differences. Similarly, Table 5 selects the 'Best NN Model' and 'Best GARCH' per index and per VaR level after inspecting the results, so the reported exceedance rates are post hoc selections and are not statistically protected. This makes the headline claim 'LSTM-SSTD performs best' and the broader viability claim weaker than stated. The authors should report all models' results without selection, correct for multiple testing, or use a genuinely separate holdout for model choice.
  3. [Section 5.3, Table 5] The comparison with classical GARCH models is not supported by the reported evidence. No estimation details, distributional assumptions beyond the labels (G, AP, E, GJR), or diagnostics for the GARCH baselines are given; no Kupiec or Christoffersen test results are reported for the GARCH models; and only four of the six indices appear in Table 5. Without these details, the claim that deep learning models 'perform competitively with classical GARCH models' cannot be verified. At minimum, the authors should provide the full set of GARCH results with the same backtests and p-values, and clarify why WIG and BOVESPA are excluded from this comparison.
  4. [Section 3.5, Eq. (20)] The PIT p-values in Table 2 are central to the calibration claim, but the paper never specifies which statistical test was applied to the PIT sequence (e.g., Kolmogorov-Smirnov, Anderson-Darling, Berkowitz). With 2,487 out-of-sample forecasts, even small deviations from uniformity can produce very small p-values. The test statistic and a discussion of effect size are necessary for the reader to interpret whether the rejections are practically significant. The exact repetition of p=2.41e-07 across multiple rows also suggests a possible reporting artifact; please report exact p-values and test statistics.
minor comments (7)
  1. [Section 3.2, Eqs. (9)-(10)] The moments for the skewed Student's t distribution are stated as E(X)=φσ and Var(X)=(γ−φ²)σ², omitting the location parameter μ. If μ is the location, the mean should be μ+φσ; if μ is the mean, the parameterization in Eq. (11) and the NLL in Eq. (16) are inconsistent. Please clarify the exact parameterization used in the implementation.
  2. [Section 4, Table 1] The walk-forward procedure is underspecified. Please clarify how the training/validation split evolves with the expanding window, how the 33% validation fraction is applied, and how the sequence length of 10 interacts with the minimum window length. Also state whether early stopping is applied per window and how the optimizer and learning-rate schedules are handled across retraining.
  3. [Table 2] The repeated PIT value 2.41e-07 for several model-index pairs is suspicious; even if the test is the same, exact equality across models with different forecast distributions is unlikely. Please report the test statistic and full precision.
  4. [Table 3] The note says 'Bolded = closest to theoretical tolerance, underlined = correct Kupiec test, * = correct Christoffersen test,' but the table as typeset does not show bold or underline, and the asterisk notation is ambiguous. Please reformat the table so the notation is legible and consistent.
  5. [Table 4] Table 4 uses commas as decimal separators (e.g., 1,263296), inconsistent with Tables 2, 3, and 5. Also, 'Chrisotff.' is a typo for 'Christoffersen.' Please standardize notation.
  6. [Section 5.1, Figure 2] The caption and text describe Figure 2 as showing DAX PIT histograms and claim the LSTM-SSTD is closest to uniform, but no p-values or test statistics are reported for DAX. Please add quantitative calibration results for all indices shown.
  7. [Section 3.3, References [5],[23]] The text says 'we apply the Fernandez and Steel [23], [5] transformation'; this is confusing because [5] and [23] are listed as different references. Please correct the citation to avoid ambiguity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the forecasting pipeline, loss functions, and evaluations are self-contained and do not reduce to their inputs.

full rationale

The paper's derivation chain is not circular. Distributional parameters are estimated by direct negative log-likelihood minimization (Eqs. 12-16) and evaluated out-of-sample via walk-forward expanding windows, LPS, CRPS, PIT, and VaR backtests. The GARCH comparison in Table 5 is an external benchmark, not an input to the neural network training or a parameter fitted to the neural network outputs. There are no load-bearing self-citations: the paper cites standard external references for scoring rules (Gneiting and Raftery), VaR tests (Kupiec, Christoffersen), and skew-t transformations (Fernandez and Steel), none of which are authored by the present author. The claim that LSTM-SSTD 'performs best' is a descriptive statement based on the reported test-set metrics, not a prediction derived from those metrics by construction. The reader's concerns about PIT calibration rejections (Table 2) and possible model-selection bias from using the test set to identify the best architecture are substantive empirical/statistical caveats, but they are not circularity: the paper does not define accuracy in terms of its own outputs, nor does it import an unverified self-cited uniqueness theorem, nor does it rename a fitted parameter as a prediction. The evaluation is self-contained and externally benchmarked, so the appropriate circularity score is 0.

Assumptions & free parameters 11 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the chosen hyperparameters and on assumptions that the distribution family, the validation scheme, and the GARCH baselines are appropriate. The hyperparameters are tuned on validation data, but the final model selection is done on the test set, which is a latent free parameter that inflates apparent performance.

free parameters (11)
  • Number of LSTM layers = 3
    Selected by manual tuning; affects model capacity.
  • Neurons per LSTM layer = 128/64/32
    Selected via KerasTuner; controls representational power.
  • Dropout rate = 0.02
    Selected via KerasTuner; regularizes training.
  • L2 regularization = 0.002
    Selected via KerasTuner; weight decay parameter.
  • Learning rate = 0.002
    Selected via KerasTuner; optimization step size.
  • Sequence length = 10
    Manual choice; number of past returns used as input.
  • Batch size = 128
    Manual choice; affects gradient estimation.
  • Epochs (early stopping) = 300
    Manual choice; training length with early stopping.
  • CNN kernel size = 2
    Manual choice; convolution window width.
  • CNN filters = 256
    Selected via KerasTuner; number of feature maps.
  • CNN pool size = 2
    Manual choice; pooling window width.
assumptions (4)
  • domain assumption The true conditional distribution of daily returns is adequately captured by one of the three parametric families: Normal, Student's t, or skewed Student's t.
    If returns follow a different distributional family, the forecast densities are misspecified and the evaluation metrics are biased.
  • domain assumption The walk-forward expanding-window procedure with single-step predictions yields unbiased out-of-sample forecasts.
    The paper relies on this to claim the test period is genuinely out-of-sample, but hyperparameter tuning details are not fully described.
  • domain assumption The GARCH models used in Section 5.3 are correctly specified and estimated, providing a valid baseline.
    No estimation details, diagnostics, or software are given for the GARCH variants, so the fairness of the comparison is assumed.
  • standard math The negative log-likelihood is a proper scoring rule for the target distribution.
    Standard maximum likelihood estimation assumes a correctly specified model; the paper uses NLL as both training loss and evaluation metric.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Forecasting Probability Distributions of Financial Returns with Deep Neural Networks." pith.science (2026). https://pith.science/paper/JUSHOIEG

@misc{pith2026250818921,
  author       = {Pith},
  title        = {Pith review of: Forecasting Probability Distributions of Financial Returns with Deep Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JUSHOIEG}},
  note         = {Machine review of arXiv:2508.18921}
}
read the original abstract

This study evaluates deep neural networks for forecasting probability distributions of financial returns. 1D convolutional neural networks (CNN) and Long Short-Term Memory (LSTM) architectures are used to forecast parameters of three probability distributions: Normal, Student's t, and skewed Student's t. Using custom negative log-likelihood loss functions, distribution parameters are optimized directly. The models are tested on six major equity indices (S\&P 500, BOVESPA, DAX, WIG, Nikkei 225, and KOSPI) using probabilistic evaluation metrics including Log Predictive Score (LPS), Continuous Ranked Probability Score (CRPS), and Probability Integral Transform (PIT). Results show that deep learning models provide accurate distributional forecasts and perform competitively with classical GARCH models for Value-at-Risk estimation. The LSTM with skewed Student's t distribution performs best across multiple evaluation criteria, capturing both heavy tails and asymmetry in financial returns. This work shows that deep neural networks are viable alternatives to traditional econometric models for financial risk assessment and portfolio management.

Figures

Figures reproduced from arXiv: 2508.18921 by the authors.

Figure 1
Figure 1. Architecture of the 1D CNN model for probabilistic forecasting. The model takes return sequences and volatility estimates as input and [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Distribution plots of PIT values for DAX index across di [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. VaR forecasts for S&P 500, Nikkei 225, and DAX indices. Blue lines show actual returns, green and red lines represent 5% and 1% [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: VaR forecasts for WIG, KOSPI, and BOVESPA indices. The models demonstrate consistent performance across di [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

23 extracted references · 21 canonical work pages

  1. [1]

    Chen, J., Lerch, S., Schienle, M., Serafin, T., and Weron, R. (2025). Probabilistic intraday electricity price forecasting using generative machine learning. arXiv preprint arXiv:2506.00044

    arXiv 2025

  2. [2]

    Chen, Y ., Kang, Y ., Chen, Y ., and Wang, Z. (2020). Probabilistic forecasting with temporal convolutional neural network.arXiv:1906.04397 [cs, stat]

    work page Pith review arXiv 2020

  3. [3]

    Christo ffersen, P. F. (1998). Evaluating interval forecasts.International Economic Review, 39(4), 841–862

    work page 1998

  4. [4]

    Duerr, O., Sick, B., and Murina, E. (2020). Probabilistic deep learning: With python, keras and tensorflow probability. Simon and Schuster

    work page 2020

  5. [5]

    and Steel, M

    Fernandez, C. and Steel, M. F. J. (1998). On Bayesian modeling of fat tails and skewness.Journal of the American Statistical Association, 93(441):359–371

    work page 1998

  6. [6]

    S., Salinas, D., Flunkert, V ., and Januschowski, T

    Gasthaus, J., Benidis, K., Wang, Y ., Rangapuram, S. S., Salinas, D., Flunkert, V ., and Januschowski, T. (2019). Probabilistic forecasting with spline quantile function rnns. In The 22nd international conference on artificial intelligence and statistics, pages 1901–1910. PMLR

    work page 2019

  7. [7]

    and Raftery, A

    Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359–378

    work page 2007

  8. [8]

    Jordan, A., Krüger, F., and Lerch, S. (2018). Evaluating probabilistic forecasts with scoringrules. arXiv:1709.04743 [cs]

    arXiv 2018

Show all 23 references
  1. [9]

    and Wang, W

    Keilbar, G. and Wang, W. (2022). Modelling systemic risk using neural network quantile regression. Empirical Economics, 62(1):93–118

    2022

  2. [10]

    Kupiec, P. (1995). Techniques for verifying the accuracy of risk measurement models. SSRN Scholarly Paper ID 6697. Rochester, NY

    1995

  3. [11]

    Marcjasz, G., Narajewski, M., Weron, R., and Ziel, F. (2023). Distributional neural networks for electricity price forecasting.Energy Economics, 125:106843

    2023

  4. [12]

    McNeil, A. J. and Frey, R. (2000). Estimation of tail-related risk measures for heteroscedastic financial time series: An extreme value approach. Journal of Empirical Finance, 7(3), 271–300

    2000

  5. [13]

    Murphy, K. P. (2012). Machine learning: A probabilistic perspective. MIT Press

    2012

  6. [14]

    Murphy, K. P. (2022). Probabilistic machine learning: An introduction. MIT Press

    2022

  7. [15]

    Murphy, K. P. (2023). Probabilistic machine learning: Advanced topics. The MIT Press

    2023

  8. [16]

    and Weron, R

    Nowotarski, J. and Weron, R. (2018). Recent advances in electricity price forecasting—a review of probabilistic forecasting.Renewable and Sustainable Energy Reviews, 81:1548–1568

    2018

  9. [17]

    Opschoor, A., van Dijk, D., and van der Wel, M. (2017). Combining density forecasts using focused scoring rules. Journal of Applied Econometrics, 32(7):1298–1313. 11

    2017

  10. [18]

    J., Ziegel, J

    Patton, A. J., Ziegel, J. F., and Chen, R. (2019). Dynamic semiparametric models for expected shortfall (and value-at-risk). Journal of Econometrics, 211(2):388–413

    2019

  11. [19]

    Salinas, D., Flunkert, V ., Gasthaus, J., and Januschowski, T. (2020). Deepar: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3):1181–1191

    2020

  12. [20]

    Teräsvirta, T. (2009). An introduction to univariate GARCH models. In T. G. Andersen, R. A. Davis, J.-P. Kreiss, and T. Mikosch (Eds.), Handbook of Financial Time Series (pp. 17-42). Springer

    2009

  13. [21]

    Thiele, S., Giacomini, R., and Patton, A. J. (2020). Modeling the conditional distribution of financial returns with asymmetric tails. Journal of Applied Econometrics, 35(6):717–742

    2020

  14. [22]

    Toubeau, J.-F., Bottieau, J., Vallée, F., and De Grève, Z. (2019). Deep learning-based multivariate probabilistic forecasting for short-term scheduling in power markets. IEEE Transactions on Power Systems, 34(2):1203–1215

    2019

  15. [23]

    and Ardia, D

    Trottier, D.-A. and Ardia, D. (2016). Moments of standardized Fernandez–Steel skewed distributions: Applications to the estimation of GARCH-type models. Finance Research Letters, 18:311–316. 12

    2016

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.