A sequential test based on ratios of scoring rules is shown to be a generalized e-value, yielding finite-sample error control for forecast method selection.
Can Two Forecasts Have the Same Conditional Expected Accuracy?
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
The approach for testing equal predictive accuracy for pairs of forecasting models proposed by Giacomini and White (2006) assumes that the parameters of the underlying forecasting models are estimated using a rolling window of fixed width and incorporates the effect of parameter estimation in the null hypothesis. We show that a necessary and sufficient condition for the conditionally expected loss differential of two forecasting models to be a martingale difference sequence is that the outcome is a simple average of the two forecasts. When the forecasts contain parameter estimation errors, this means that the conditional mean of the outcome has to be a function of past estimation errors--a condition that fails in many situations. We also show that the null can fail even in the absence of parameter estimation for many types of stochastic processes in common use.
fields
math.ST 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Sequential Scoring Rule Evaluation for Forecast Method Selection
A sequential test based on ratios of scoring rules is shown to be a generalized e-value, yielding finite-sample error control for forecast method selection.