Solved – How to reconcile how EViews estimates an AR(1) model with OLS

armaeconometricsestimationforecastingleast squares

I discuss estimation of the following $ARX(1)$ model:

$Y_t=α+βX_t+u_t$ where $u_t=ρu_{t-1}+ϵ_t$

Substituting the the value of $u_t$ in the first equation into the second, we have that:

$Y_t-α- βX_t =ρ(Y_{t-1}-α-βX_{t-1})+ϵ_t$

Rearranging, we have:

$Y_t=α(1-ρ)+ρY_{t-1}+ βX_t-ρβX_{t-1}+ϵ_t$

Can we obtain estimates for the parameters in equation 2 using OLS? Well, yes. However, notice that if we did this, we’d be ignoring the fact that the parameters in the model are restricted, in particular the parameter of $X_{t-1}$, $- ρβ$, is the product of the coefficients on $Y_{t-1}$ and $X_{t}$. We should utilise this information. The OLS estimator will be unbiased, consistent, and BLUE (best linear unbiased estimator), but by utilising the information regarding the restriction of parameters, we can find a better estimator than OLS. In other words, the non-linear estimator produced by the Marquardt algorithm will be superior to OLS. Unsurprisingly, EViews estimates all ARMAX models using the Marquardt algorithm.

Consider the following 3 situations:

$1.$ If $ρ=0$, then our model becomes:

$Y_t=α+ βX_t+ϵ_t$

The best thing to do in this case is to ignore $Y_{t-1}$ and $X_{t-1}$, since we know, in theory, they are irrelevant in explaining $Y_{t}$. The Marquardt algorithm is not necessary.

$2.$ If $β=0$ and $ρ ≠1$, then our model becom$Y_t=α(1-ρ)+ρY_{t-1}+ϵ_t$

The best thing to do in this case is to ignore $X_t$ and $X_{t-1}$ since we know they are irrelevant variables.

$3.$ If, in the population, $X_{t-1}$ is uncorrelated with the other regressors, in particular $X_{t}$, then we can just regress $Y_t$ on a constant, $Y_{t-1}$, $X_{t}$. Again, this is extremely unlikely. Especially so because we are dealing with time series variables that are in all but the most rare circumstances autocorrelated. If, in the sample, $X_{t-1}$ is uncorrelated with the other regressors, a very unlikely scenario, then the OLS estimates of the $α(1-ρ)$, $ρ$ and $β$ when we ignore $X_{t-1}$ are exactly the same as from the Marquardt algorithm, by the Frisch-Waugh-Lovell theorem, as long as the estimate of $ρ$ is not precisely zero (which is, in any case, unlikely). We can obtain the estimate of $α$ from the Marquardt algorithm by simply multiplying the OLS estimate of the constant by 1 minus the OLS estimate of $ρ$.

This is not so much a question. I would like your expertise regarding whether you think my assertions are correct. It is very difficult to find answers to this online. It is also quite an interesting problem.

Your help is greatly appreciated. Thank you.

Christian

Best Answer

I assume when you talk about "doing some algebra", you mean that you will simply run a regression of Y = a + bY(-1) + cX + dX(-1), then saying that alpha = a / (1-b).

This cannot be done. You're ignoring the restrictions on rho. Specifically, you're ignoring the fact that rho is also part of the coefficient on X(-1).

There is no way, once you have Xs involved, to switch from the OLS to the ARMA model. If there were, software packages would use OLS!

Related Solutions

Solved – GLS estimator that has lower variance than OLS for sum of parameters in linear model under Gauss-Markov conditions

The Gauss-Markov theorem states that the covariance matrix of any unbiased estimator $\tilde{\beta} \ne \hat{\beta}_{OLS}$ exceeds that of $\hat{\beta}_{OLS}$ by a positive semidefinite matrix. Let's label the OLS covariance matrix $\Omega$ and the positive semidefinite matrix $D$. The variance of the sum of the OLS estimates can be written as $\iota' \Omega \iota$, where $\iota$ is a vector of ones of the appropriate length. For a non-OLS estimator, the variance of the sum is:

$\sigma^2_\Sigma = \iota' (\Omega + D) \iota = \iota' \Omega \iota + \iota' D \iota \geq \iota' \Omega \iota$

as $D$ is positive semidefinite. Therefore, the sum of the OLS parameter estimates is the minimum variance unbiased estimator of the true sum of the parameters.

In fact, this applies to any weighted sum of the parameter estimates, not just the unweighted sum.

Solved – Other unbiased estimators than the BLUE (OLS solution) for linear models

One example that comes to mind is some GLS estimator that weights observations differently although that is not necessary when the Gauss-Markov assumptions are met (which the statistician may not know to be the case and hence apply still apply GLS).

Consider the case of a regression of $y_i$, $i=1,\ldots,n$ on a constant for illustration (readily generalizes to general GLS estimators). Here, $\{y_i\}$ is assumed to be a random sample from a population with mean $\mu$ and variance $\sigma^2$.

Then, we know that OLS is just $\hat\beta=\bar y$, the sample mean. To emphasize the point that each observation is weighted with weight $1/n$, write this as $$ \hat\beta=\sum_{i=1}^n\frac{1}{n}y_i. $$ It is well-known that $Var(\hat\beta)=\sigma^2/n$.

Now, consider another estimator which can be written as $$ \tilde\beta=\sum_{i=1}^nw_iy_i, $$ where the weights are such that $\sum_iw_i=1$. This ensures that the estimator is unbiased, as $$ E\left(\sum_{i=1}^nw_iy_i\right)=\sum_{i=1}^nw_iE(y_i)=\sum_{i=1}^nw_i\mu=\mu. $$ Its variance will exceed that of OLS unless $w_i=1/n$ for all $i$ (in which case it will of course reduce to OLS), which can for instance be shown via a Lagrangian:

\begin{align*} L&=V(\tilde\beta)-\lambda\left(\sum_iw_i-1\right)\\ &=\sum_iw_i^2\sigma^2-\lambda\left(\sum_iw_i-1\right), \end{align*} with partial derivatives w.r.t. $w_i$ set to zero being equal to $2\sigma^2w_i-\lambda=0$ for all $i$, and $\partial L/\partial\lambda=0$ equaling $\sum_iw_i-1=0$. Solving the first set of derivatives for $\lambda$ and equating them yields $w_i=w_j$, which implies $w_i=1/n$ minimizes the variance, by the requirement that the weights sum to one.

Here is a graphical illustration from a little simulation, created with the code below:

EDIT: In response to @kjetilbhalvorsen's and @RichardHardy's suggestions I also include the median of the $y_i$, the MLE of the location parameter pf a t(4) distribution (I get warnings that In log(s) : NaNs produced that I did not check further) and Huber's estimator in the plot.

We observe that all estimators seem to be unbiased. However, the estimator that uses weights $w_i=(1\pm\epsilon)/n$ as weights for either half of the sample is more variable, as are the median, the MLE of the t-distribution and Huber's estimator (the latter only slightly so, see also here).

That the latter three are outperformed by the OLS solution is not immediately implied by the BLUE property (at least not to me), as it is not obvious if they are linear estimators (nor do I know if the MLE and Huber are unbiased).

library(MASS)
n <- 100      
reps <- 1e6

epsilon <- 0.5
w <- c(rep((1+epsilon)/n,n/2),rep((1-epsilon)/n,n/2))

ols <- weightedestimator <- lad <- mle.t4 <- huberest <- rep(NA,reps)

for (i in 1:reps)
{
  y <- rnorm(n)
  ols[i] <- mean(y)
  weightedestimator[i] <- crossprod(w,y)  
  lad[i] <- median(y)   
  mle.t4[i] <- fitdistr(y, "t", df=4)$estimate[1]
  huberest[i] <- huber(y)$mu
}

plot(density(ols), col="purple", lwd=3, main="Kernel-estimate of density of OLS and other estimators",xlab="")
lines(density(weightedestimator), col="lightblue2", lwd=3)     
lines(density(lad), col="salmon", lwd=3)     
lines(density(mle.t4), col="green", lwd=3)
lines(density(huberest), col="#949413", lwd=3)
abline(v=0,lty=2)
legend('topright', c("OLS","weighted","median", "MLE t, 4 df", "Huber"), col=c("purple","lightblue","salmon","green", "#949413"), lwd=3)

Best Answer

Related Solutions

Solved – GLS estimator that has lower variance than OLS for sum of parameters in linear model under Gauss-Markov conditions

Solved – Other unbiased estimators than the BLUE (OLS solution) for linear models

Related Question