Solved – Unbiased estimator for AR($p$) model

autoregressivemaximum likelihoodtime seriesunbiased-estimator

Consider an AR($p$) model (assuming zero mean for simplicity):

$$ x_t = \varphi_1 x_{t-1} + \dotsc + \varphi_p x_{t-p} + \varepsilon_t $$

The OLS estimator (equivalent to the conditional maximum likelihood estimator) for $\mathbf{\varphi} := (\varphi_1,\dotsc,\varphi_p)$ is known to be biased, as noted in a recent thread.

(Curiously, I could neither find the bias mentioned in Hamilton "Time Series Analysis" nor in a few other time series textbooks. However, it can be found in various lecture notes and academic articles, e.g. this.)

I was not able to find out whether the exact maximum likelihood estimator of AR($p$) is biased or not; hence my first question.

  • Question 1: Is exact maximum likelihood estimator of AR($p$) model's autoregressive parameters $\varphi_1,\dotsc,\varphi_p$ biased? (Let us assume the AR($p$) process is stationary. Otherwise the estimator is not even consistent, since it is restricted in the stationary region; see, e.g., Hamilton "Time Series Analysis", p. 123.)

Also,

  • Question 2: Are there any reasonably simple unbiased estimators?

Best Answer

This is of course not a rigorous answer to your question 1, but since you asked the question in general, evidence for a counterexample already indicates that the answer is no.

So here is a little simulation study using exact ML estimation from arima0 to argue that there is at least one case where there is bias:

reps <- 10000
n <- 30
true.ar1.coef <- 0.9

ar1.coefs <- rep(NA, reps)
for (i in 1:reps){
  y <- arima.sim(list(ar=true.ar1.coef), n)
  ar1.coefs[i] <- arima0(y, order=c(1,0,0), include.mean = F)$coef
}
mean(ar1.coefs) - true.ar1.coef
Related Question