Say one has finished estimating a correctly specified GARCH(1,1) on a daily time series and now wants to evaluate the accuracy of the one step ahead forecasts what steps or tests could one do?
I understand that the MSE
$$MSE = \frac{1}{N} RSS = \frac{1}{N} \sum (\hat{\sigma}_i -\sigma_i)^2$$
can be computed where $N$ is the number of samples and $\hat{\sigma}_i$ is the estimated one step ahead volatility. Because we do not know the realized volatility $\sigma_i$ we can use the squared return of that day as proven here.
But is the one step ahead predictor not already defined as the value $\hat{\sigma}$ of the volatility that minimizes the MSE? So why compute this measure if it going to be the minimum across models anyway?
Moreover do I understand correctly that the practical way to compute the $MSE$
for the one step ahead forecast is:
- Estimate the correctly specified model on your data (returns) except the last data point.
- Compute the one step ahead forecast of your model.
- Repeat step 1 but take away an other data point.
Doing this process $N$ times one obtains $N$ $\hat{\sigma}_i$ that can then by utilized in the given formula. Am I getting this right?
Also I understand from this paper (Bollerslev 1998) that utilizing the squared daily return to approximate the realized volatility leads to noise.
For example a better estimate of realized daily volatility would be the sum of 30 minutes squared returns of that day. Is this correct?
So to get a "better" $MSE$ I could substitute every $\sigma_i$ with the sum of 30 minutes squared returns of that day instead of simply the daily squared return? Is this how you use realized volatility to evaluate the goodness of your forecasts?
I will end this rambling by asking for a good reference in evaluating the accuracy of the forecasts using realized volatility because it is obvious that I am very confused. I can't seem to find a good text on my own.
Because I have asked a lot of questions in a confused fashion I will put a bounty of 50 reputation so you might want to wait to answer (but you could comment to give me a reference straight away if you happen to know one).
Best Answer
If you estimate the GARCH model using maximum likelihood then the fitted values $\hat\sigma_t$ are the likelihood-maximizing values (subject to the GARCH(1,1) functional form) which need not coincide with MSE-minimizing values. That depends on the distribution assumed for the likelihood calculation.
Also, when fitting the model on a data sample indexed from $1$ to $T$, the fitted value $\hat\sigma^2_t$ for $t<T$ utilizes the information not only from $1\leqslant\tau<t$ but also from $t\leqslant\tau\leqslant T$. Meanwhile, to fairly evaluate the forecast performance, you should not allow the estimate to be calculated using future data; instead, you need to check the forecast accuracy out of sample.
Because you may want to know what the actual values of MSE is. Knowing that the MSE is minimal does not tell you what its value is. (See also the answer to the previous question.)
That will be the mean squared forecast error. If you want in-sample MSE, just use the fitted values from the model estimated on the whole sample. The former should give an unbiased estimate of model performance, while the latter will be too optimistic in that respect.
Very important point indeed! The response variable of the GARCH model is measured with noise when squared errors are used as proxies; this noise may be quite substantial. When trying to assess model fit, the measurement error associated with the dependent variable may cause quite some trouble.
On the first thought, that could be a valid option. If that was proposed in the Andersen and Bollerslev (1998) paper, then it must be fine.
That makes sense to me.
Regarding references, I think the Andersen and Bollerslev (1998) paper is quite relevant and complete. Also see Patton & Sheppard "Evaluating volatility and correlation forecasts" (2009) (free version here) and other works by Patton.