Suppose a fit a line using the method of least squares to $n$ points, all the standard statistical assumptions hold, and I want to estimate that line at a new point, $x_0$. Denoting that value by $\hat{y_0}$, the estimate is given by:
$$\hat{y_0} = \hat{\beta_0} + \hat{\beta_1}x_0$$
Where $$\hat{\beta_0} = \bar{y} – \hat{\beta_1}\bar{x}$$ and $$\hat{\beta_1} = \frac{\sum_{i=1}^n (x_i-\bar{x})(y_i – \bar{y})}{\sum_{i=1}^n (x_i – \bar{x})^2}$$
I now want an expression for the variance of $\hat{y_0}$. I tried to do it as follows:
$$V(\hat{y_0}) = V(\hat{\beta_0}) + V(\hat{\beta_1}x_0) + 2Cov(\hat{\beta_0},\hat\beta_1x_0)$$
Where $V(\hat{\beta_0}) = 0$ since $\beta_0$ is constant by definition (depending on the sample means). I also think that the covariance term should be zero because $E(\hat{\beta_0}\hat{\beta_1x_0}) = E(\hat\beta_0)(\hat\beta_1x_0)$ intuitively, but I haven't proven this. I'm not sure what to do about the middle term — do I just hammer out the computation?
Advice/Solution would be appreciated. Note: this is a review question for a test I have tomorrow, not homework. I need to understand this.
Best Answer
Actually, $\hat{\beta}_0$ isn't constant - it depends on the $y_i$ (which follow a normal distribution). Note that in most cases, this variance would be likely computed for a prediction interval since you're working with a new $x$-value.
Using properties of variances and covariances, $$\begin{align} \newcommand{\Var}[1]{\text{Var}\left(#1\right)}\newcommand{\Cov}[2]{\text{Cov}\left(#1, #2\right)}\Var{y_0} &= \Var{\hat{\beta}_0}+\Var{\hat{\beta}_1x_0} + 2\Cov{\hat{\beta}_0}{\hat{\beta}_1x_0} \\ &= \Var{\hat{\beta}_0}+x_0^2\Var{\hat{\beta}_1}+2x_0\Cov{\hat{\beta}_0}{\hat{\beta}_1} \end{align}$$ Now recall that the $y_i$ (not the predicted values) are based on $$y_i = \beta_0+\beta_1x_i+\epsilon_i\text{, } \epsilon_i \overset{iid}{\sim} \mathcal{N}(0, \sigma^2)$$
You can derive $\Var{\hat{\beta}_0}$ using the above, as here. You can also see here the derivation of $\Var{\hat{\beta}_1}$.
I haven't been able to find a derivation of the covariance. One way you could do this is by using $$\Cov{\hat{\beta}_0}{\hat{\beta}_1} = \Cov{\bar{y}-\hat{\beta}_1\bar{x}}{\hat{\beta}_1} = \Cov{\bar{y}}{\hat{\beta}_1} - \bar{x}\Var{\hat{\beta}_1}$$ and this might be helpful.
The derivation is very, very tedious and long, so I wouldn't expect to see this on an exam.
Unfortunately, it's been a long time since I've done these derivations, and I'm used to doing them using matrices (which, quite frankly, is a lot cleaner).