Heavy Tails and Predictive Ability Testing
Pith reviewed 2026-05-20 15:31 UTC · model grok-4.3
The pith
The Diebold-Mariano test rejects a true null as often as 70 percent of the time when loss differentials have infinite variance.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
When loss differentials have infinite variance, the Diebold-Mariano test statistic converges to a nonstandard limit involving non-Gaussian stable random variables. As a consequence, conventional critical values can yield severely distorted inference: a nominal 5% test may reject a true null as often as 70% of the time. To establish these results, a new stable limit theorem is developed for strongly mixing, infinite-variance time series processes. Building on this theory, sub-sampling-based inference is shown to remain valid irrespective of tail-heaviness and requires no estimation of long-run variances or tail indices.
What carries the argument
New stable limit theorem for strongly mixing infinite-variance time series processes that justifies subsampling inference for the Diebold-Mariano statistic without variance or tail-index estimation.
If this is right
- Accounting for heavy tails can substantially alter conclusions about which forecast performs better in applications such as risk forecasting for exchange rates.
- Subsampling provides valid tests without any need to estimate long-run variances or tail indices.
- The size distortion affects any comparison that relies on the Diebold-Mariano statistic whenever tails are regularly varying with index in (1,2).
- Standard normal-based procedures become unreliable for predictive-ability testing under infinite variance.
Where Pith is reading between the lines
- Many existing studies that compare economic or financial forecasts may require re-evaluation if the loss differentials exhibit heavy tails.
- The subsampling method could be extended to other forecast-evaluation statistics that share the same limiting behavior under infinite variance.
- Practitioners should examine the tail properties of their loss series before interpreting conventional test results.
Load-bearing premise
Loss differentials form a strongly mixing sequence whose tails are regularly varying with index between 1 and 2.
What would settle it
Generate loss differential series from a stable distribution with tail index 1.5, apply the Diebold-Mariano test using standard normal critical values, and observe whether rejection rates stay near 5 percent or rise substantially higher.
Figures
read the original abstract
We study the asymptotic behaviour of widely used tests for evaluating and comparing predictive accuracy when forecast errors exhibit heavy tails. In particular, when loss differentials have infinite variance, the Diebold-Mariano test statistic converges to a nonstandard limit involving non-Gaussian stable random variables. As a consequence, conventional critical values can yield severely distorted inference: a nominal 5$\%$ test may reject a true null as often as 70$\%$ of the time. To establish these results, we develop a new stable limit theorem for strongly mixing, infinite-variance time series processes. Building on this theory, we consider sub-sampling-based inference that remains valid irrespective of tail-heaviness and requires no estimation of long-run variances or tail indices. An application to risk forecasts for emerging-market exchange rates shows that accounting for heavy tails can substantially alter conclusions about predictive performance relative to standard procedures.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript studies the asymptotic behavior of the Diebold-Mariano test for predictive accuracy when loss differentials exhibit heavy tails and infinite variance. It establishes a new stable limit theorem for strongly mixing sequences whose tails are regularly varying with index in (1,2), showing convergence of the test statistic to a non-Gaussian stable random variable. This implies that conventional critical values produce severe size distortions, with a nominal 5% test rejecting a true null as often as 70% of the time. The authors propose a subsampling procedure that delivers valid inference without requiring consistent estimation of long-run variance or tail index, and apply the method to risk forecasts for emerging-market exchange rates.
Significance. If the limit theorem and its consequences hold, the paper addresses a practically relevant gap in forecast evaluation methods used in economics and finance, where heavy tails are common. The extension of stable-limit results to dependent infinite-variance processes and the accompanying subsampling approach that avoids nuisance-parameter estimation constitute a clear methodological contribution. The empirical illustration demonstrates that accounting for heavy tails can change substantive conclusions about model performance.
major comments (2)
- [Theorem 2.1] Theorem 2.1 (stable limit theorem): the normalization sequence and the form of the limiting stable law are stated, but the proof does not explicitly verify that the strong-mixing coefficients decay fast enough to preserve the domain-of-attraction condition for the triangular array of partial sums when the tail index lies in (1,2).
- [Section 4.1] Section 4.1, size-distortion calculation: the 70% rejection probability for the nominal 5% test is obtained from the cdf of the limiting stable distribution; the paper should report the exact tail index and skewness parameter used to obtain this figure and confirm that it is not an upper bound attained only at the boundary α=1+.
minor comments (2)
- [Equation (8)] The notation for the stable characteristic function in Equation (8) uses an unconventional parameterization; aligning it with the standard form in Nolan (2020) would improve readability.
- [Table 2] Table 2 reports empirical rejection frequencies but does not include results for the subsampling procedure under the same heavy-tailed designs; adding these columns would strengthen the comparison.
Simulated Author's Rebuttal
We thank the referee for the positive evaluation and constructive comments, which have helped us improve the manuscript. We address each major comment below.
read point-by-point responses
-
Referee: [Theorem 2.1] Theorem 2.1 (stable limit theorem): the normalization sequence and the form of the limiting stable law are stated, but the proof does not explicitly verify that the strong-mixing coefficients decay fast enough to preserve the domain-of-attraction condition for the triangular array of partial sums when the tail index lies in (1,2).
Authors: We appreciate the referee highlighting this point for added clarity. The proof of Theorem 2.1 invokes the general stable convergence result for strongly mixing sequences with regularly varying tails (drawing on standard results such as those in the literature on domain-of-attraction conditions for dependent triangular arrays). To make the verification explicit, we have inserted a short remark immediately following the statement of Theorem 2.1 that confirms the mixing rate α(n) = O(n^{-r}) with r > 1 is sufficient to preserve the required conditions when the tail index lies in (1,2). revision: yes
-
Referee: [Section 4.1] Section 4.1, size-distortion calculation: the 70% rejection probability for the nominal 5% test is obtained from the cdf of the limiting stable distribution; the paper should report the exact tail index and skewness parameter used to obtain this figure and confirm that it is not an upper bound attained only at the boundary α=1+.
Authors: We thank the referee for this suggestion. The reported figure of approximately 70% is obtained from the cdf of a symmetric (β = 0) stable distribution with tail index α = 1.5, a value representative of many financial loss differentials. We have revised Section 4.1 to state these parameters explicitly and to note that the size distortion is not an upper bound attained only at the boundary; the rejection probability increases as α approaches 1 from above but remains substantial throughout the interval (1,2). A brief sensitivity table for α ∈ {1.2, 1.5, 1.8} has also been added. revision: yes
Circularity Check
No significant circularity; derivation self-contained
full rationale
The paper's central contribution is a new stable limit theorem for the Diebold-Mariano statistic under strong mixing and regular variation of tails with index in (1,2). This is derived from standard assumptions on the loss differentials and extends existing results for dependent heavy-tailed sequences without reducing to self-definitional equations, fitted parameters renamed as predictions, or load-bearing self-citations. The reported size distortions follow directly from the tail properties of the non-Gaussian stable limit, and the subsampling inference is constructed to be valid without requiring consistent variance or tail-index estimation. No step in the provided abstract or described chain equates a claimed result to its own inputs by construction; the argument remains independent of the target conclusions.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Loss differentials form a strongly mixing sequence with regularly varying tails of index alpha in (1,2)
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
Theorem 2.6 ... a_n^{-1} (S_n, γ_n) → (ξ_κ, ζ_{κ/2}) ... hybrid characteristic function–Laplace transform (2.14)
-
IndisputableMonolith/Foundation/RealityFromDistinction.leanreality_from_one_distinction unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
Assumptions 2–3 (regular variation + anti-clustering + mixing rates)
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Bai, S., M. S. Taqqu, and T. Zhang (2016): A unified approach to self-normalized block sampling, Stochastic Processes and their Applications, 126, 2465--2493
work page 2016
-
[2]
Barendse, S. and A. J. Patton (2022): Comparing predictive accuracy in the presence of a loss function shape parameter, Journal of Business & Economic Statistics, 40, 1057--1069
work page 2022
-
[3]
Bartkiewicz, K., A. Jakubowski, T. Mikosch, and O. Wintenberger (2011): Stable limits for sums of dependent infinite variance random variables, Probability Theory and Related Fields, 150, 337--372
work page 2011
-
[4]
Basrak, B. and J. Segers (2009): Regularly varying multivariate time series, Stochastic Processes and their Applications, 119, 1055--1080
work page 2009
-
[5]
Buraczewski, D., E. Damek, and T. Mikosch (2016): Stochastic Models with Power-Law Tails: The Equation X=AX+B , Switzerland: Springer
work page 2016
-
[6]
Christensen, B. J. and R. T. Varneskov (2021): Dynamic global currency hedging, Journal of Financial Econometrics, 19, 97--127
work page 2021
-
[7]
Davis, R. A. (1983): Stable limits for partial sums of dependent random variables, The Annals of Probability, 11, 262--269
work page 1983
-
[8]
Davis, R. A., H. Drees, J. Segers, and M. Warchoł (2018): Inference on the tail process with application to financial time series modeling, Journal of Econometrics, 205, 508--525
work page 2018
-
[9]
Davis, R. A. and T. Hsing (1995): Point process and partial sum convergence for weakly dependent random variables with infinite variance, The Annals of Probability, 23, 879--917
work page 1995
-
[10]
Diebold, F. X. and R. S. Mariano (1995): Comparing predictive accuracy, Journal of Business & Economic Statistics, 13, 253--263
work page 1995
-
[11]
(1994): Mixing: Properties and Examples, Lecture Notes in Statistics, New York, NY: Springer-Verlag
Doukhan, P. (1994): Mixing: Properties and Examples, Lecture Notes in Statistics, New York, NY: Springer-Verlag
work page 1994
-
[12]
Fern \'a ndez, C. and M. F. Steel (1998): On Bayesian modeling of fat tails and skewness, Journal of the American Statistical Association, 93, 359--371
work page 1998
-
[13]
(2009): Power laws in economics and finance, Annual Review of Economics, 1, 255--93
Gabaix, X. (2009): Power laws in economics and finance, Annual Review of Economics, 1, 255--93
work page 2009
-
[14]
Gabaix, X., P. Gopikrishnan, V. Plerou, and H. E. Stanley (2006): Institutional investors and stock market volatility, The Quarterly Journal of Economics, 121, 461--504
work page 2006
-
[15]
Giacomini, R. and I. Komunjer (2005): Evaluation and combination of conditional quantile forecasts, Journal of Business & Economic Statistics, 23, 416--431
work page 2005
-
[16]
Hansen, P. R. (2005): A test for superior predictive ability, Journal of Business & Economic Statistics, 23, 365--380
work page 2005
-
[17]
Hansen, P. R. and A. Lunde (2005): A forecast comparison of volatility models: does anything beat a GARCH(1,1)? Journal of Applied Econometrics, 20, 873--889
work page 2005
-
[18]
Hansen, P. R., A. Lunde, and J. M. Nason (2011): The model confidence set, Econometrica, 79, 453--497
work page 2011
-
[19]
Harvey, D. I., S. J. Leybourne, and P. Newbold (1998): Tests for forecast encompassing, Journal of Business & Economic Statistics, 16, 254--259
work page 1998
-
[20]
Ibragimov, I. A. (1962): Some limit theorems for stationary processes, Theory of Probability & Its Applications, 7, 349--382
work page 1962
-
[21]
Ibragimov, M., R. Ibragimov, and P. Kattuman (2013): Emerging markets and heavy tails, Journal of Banking & Finance, 37, 2546--2559
work page 2013
-
[22]
Ibragimov, M., R. Ibragimov, and J. Walden (2015): Heavy-Tailed Distributions and Robustness in Economics and Finance, Lecture Notes in Statistics, Heidelberg: Springer
work page 2015
-
[23]
Kim, J., N. Meddahi, and M. Yamashita (2021): Forecast comparison tests under fat-tails, Working paper accessed on 24 November 2025 via https://www.ecb.europa.eu/press/conferences/shared/pdf/20210615_11th_cft/Yamashita_paperT11.pdf
-
[24]
Kokoszka, P. and M. Wolf (2004): Subsampling the mean of heavy-tailed dependent observations, Journal of Time Series Analysis, 25, 217--234
work page 2004
-
[25]
Li, J., Z. Liao, and R. Quaedvlieg (2021): Conditional superior predictive ability, The Review of Economic Studies, 89, 843--875
work page 2021
-
[26]
Logan, B. F., C. L. Mallows, S. O. Rice, and L. A. Shepp (1973): Limit distributions of self-normalized sums, The Annals of Probability, 1, 788--809
work page 1973
-
[27]
Matsui, M., T. Mikosch, and O. Wintenberger (2025 a ): Moments for self-normalized partial sums, Stochastic Processes and their Applications, 198, 104810
work page 2025
-
[28]
--- -.1pt --- -.1pt --- (2025 b ): Self-normalized partial sums of heavy-tailed time series, Stochastic Processes and their Applications, 190, 104729
work page 2025
-
[29]
McElroy, T. and D. N. Politis (2002): Robust inference for the mean in the presence of serial correlation and heavy-tailed distributions, Econometric Theory, 18, 1019--1039
work page 2002
-
[30]
Mikosch, T. and O. Wintenberger (2024): Extreme Value Theory for Time Series: Models with Power-Law Tails, Heidelberg: Springer
work page 2024
-
[31]
Newey, W. K. and K. D. West (1987): A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix, Econometrica, 55, 703--708
work page 1987
-
[32]
--- -.1pt --- -.1pt --- (1994): Automatic lag selection in covariance matrix estimation, The Review of Economic Studies, 61, 631--653
work page 1994
-
[33]
Odendahl, F., B. Rossi, and T. Sekhposyan (2023): Evaluating forecast performance with state dependence, Journal of Econometrics, 237, 105220
work page 2023
-
[34]
Patton, A. J. and A. Timmermann (2012): Forecast rationality tests based on multi-horizon bounds, Journal of Business & Economic Statistics, 30, 1--17
work page 2012
-
[35]
Patton, A. J., J. F. Ziegel, and R. Chen (2019): Dynamic semiparametric models for expected shortfall (and Value-at-Risk), Journal of Econometrics, 211, 388--413
work page 2019
-
[36]
Politis, D. N., J. P. Romano, and M. Wolf (1999): Subsampling, New York, NY: Springer
work page 1999
-
[37]
Samorodnitsky, G. and M. S. Taqqu (1994): Stable Non-Gaussian Random Processes: Stochastic Models with Infinite Variance, Boca Raton, FL: Chapman & Hall
work page 1994
-
[38]
(2000): A reality check for data snooping, Econometrica, 68, 1097--1126
White, H. (2000): A reality check for data snooping, Econometrica, 68, 1097--1126
work page 2000
-
[39]
Zhang, T., H.-C. Ho, M. Wendler, and W. B. Wu (2013): Block sampling under strong dependence, Stochastic Processes and their Applications, 123, 2323--2339
work page 2013
-
[40]
Journal of Business & Economic Statistics , volume=
Comparing predictive accuracy , author=. Journal of Business & Economic Statistics , volume=
- [41]
- [42]
-
[43]
Stochastic Processes and their Applications , volume=
A unified approach to self-normalized block sampling , author=. Stochastic Processes and their Applications , volume=. 2016 , publisher=
work page 2016
- [44]
-
[45]
An Introduction to Probability Theory and Its Applications , author=. 1971 , publisher=
work page 1971
-
[46]
Self-normalized partial sums of heavy-tailed time series , journal =
Muneya Matsui and Thomas Mikosch and Olivier Wintenberger , keywords =. Self-normalized partial sums of heavy-tailed time series , journal =. 2025 , issn =
work page 2025
-
[47]
Journal of Time Series Analysis , volume=
Subsampling the mean of heavy-tailed dependent observations , author=. Journal of Time Series Analysis , volume=. 2004 , publisher=
work page 2004
-
[48]
The Annals of Probability , volume=
Point process and partial sum convergence for weakly dependent random variables with infinite variance , author=. The Annals of Probability , volume=. 1995 , publisher=
work page 1995
-
[49]
Journal of Empirical Finance , volume=
Testing the covariance stationarity of heavy-tailed time series: An overview of the theory with applications to several financial datasets , author=. Journal of Empirical Finance , volume=. 1994 , publisher=
work page 1994
-
[50]
Extreme Value Theory for Time Series: Models with Power-Law Tails , author=. 2024 , publisher=
work page 2024
- [51]
-
[52]
Stochastic Processes and their Applications , volume=
Regularly varying multivariate time series , author=. Stochastic Processes and their Applications , volume=. 2009 , publisher=
work page 2009
-
[53]
Stochastic Models with Power-Law Tails: The Equation X=AX+B , author=. 2016 , publisher=
work page 2016
-
[54]
Asymptotic Theory of Weakly Dependent Random Processes , author=. 2017 , publisher=
work page 2017
-
[55]
Probability Theory and Related Fields , volume=
Stable limits for sums of dependent infinite variance random variables , author=. Probability Theory and Related Fields , volume=. 2011 , publisher=
work page 2011
- [56]
-
[57]
Annual Review of Economics , volume=
Power laws in economics and finance , author=. Annual Review of Economics , volume=
-
[58]
Heavy-Tailed Distributions and Robustness in Economics and Finance , author =. 2015 , publisher=
work page 2015
- [59]
- [60]
-
[61]
Journal of Applied Econometrics , volume =
Coroneo, Laura and Iacone, Fabrizio , title =. Journal of Applied Econometrics , volume =
-
[62]
Francis X. Diebold , title =. Journal of Business & Economic Statistics , volume =
-
[63]
Donald W. K. Andrews , journal =. Heteroskedasticity and autocorrelation consistent covariance matrix estimation , volume =
-
[64]
Whitney K. Newey and Kenneth D. West , journal =. A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix , volume =
-
[65]
arXiv preprint arXiv:2503.15894 , year=
The Gaussian central limit theorem for a stationary time series with infinite variance , author=. arXiv preprint arXiv:2503.15894 , year=
-
[66]
Mixing and moment properties of various GARCH and stochastic volatility models , volume=. Econometric Theory , author=. 2002 , pages=. doi:10.1017/S0266466602181023 , number=
-
[67]
David I. Harvey and Stephen J. Leybourne and Paul Newbold , journal =. Tests for forecast encompassing , urldate =
- [68]
-
[69]
Proceedings of the national Academy of Sciences , volume=
A central limit theorem and a strong mixing condition , author=. Proceedings of the national Academy of Sciences , volume=
-
[70]
Stable Non-Gaussian Random Processes: Stochastic Models with Infinite Variance , author=. 1994 , publisher=
work page 1994
- [71]
-
[72]
Journal of Business & Economic Statistics , volume =
Peter Reinhard Hansen , title =. Journal of Business & Economic Statistics , volume =
-
[73]
Evaluating forecast performance with state dependence , journal =. 2023 , issn =. doi:https://doi.org/10.1016/j.jeconom.2021.07.015 , url =
-
[74]
and Lunde, Asger and Nason, James M
Hansen, Peter R. and Lunde, Asger and Nason, James M. , title =. Econometrica , volume =
- [75]
-
[76]
Robust inference for the mean in the presence of serial correlation and heavy-tailed distributions , volume=. Econometric Theory , author=. 2002 , pages=. doi:10.1017/S026646660218501X , number=
-
[77]
Patrice Bertail and Dimitris N. Politis and Joseph P. Romano , title =. Journal of the American Statistical Association , volume =. 1999 , publisher =
work page 1999
-
[78]
Testing for equal predictive accuracy with strong dependence , journal =
Laura Coroneo and Fabrizio Iacone , keywords =. Testing for equal predictive accuracy with strong dependence , journal =. 2025 , issn =. doi:https://doi.org/10.1016/j.ijforecast.2024.11.003 , url =
-
[79]
Arthur Berg and Timothy L. McMurry and Dimitris N. Politis , abstract =. Subsampling p-values , journal =. 2010 , issn =. doi:https://doi.org/10.1016/j.spl.2010.04.018 , url =
-
[80]
Sander Barendse and Andrew J. Patton , title =. Journal of Business & Economic Statistics , volume =. 2022 , publisher =
work page 2022
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.