Solved – Variance of slope

rregressionregression coefficientsvariance

I have a bunch of data that I fit a linear regression to, and now I need to find the variance of my slope. Is there an analytical way to get this?

If an example is necessary, consider this my data in R:

x <- c(1:6)
y <- c(18, 14, 15, 12, 7, 6)
lm(y ~ x)$coefficients

So I have a slope estimate of -2.4, but I want to know the variance of that estimate.

After looking at previous questions, I've seen a few equations for estimating the slope parameter, but I'm a little confused about what the differences between equations are and what approach is valid for my problem.

For example, the answers in this question say that $\newcommand{\Var}{\rm Var}\newcommand{\slope}{\rm slope}\Var[\slope] = \frac{V[Y]}{\sum\left(\frac{x_i-\bar{x}}{\sum(x_i-\bar{x})^2}\right)}$.

This question says that $\Var[\slope] = \frac{V[Y]}{\sum(x_i-\bar{x})^2}$.

And if I look at the output in R (as a "check" mechanism), I'm given two other ways I could potentially calculate the slope variance (one using the standard error, another given the covariance matrix). I feel like I'm missing something key because all these estimates give me similar (but not the same) answer.

Best Answer

For a standard linear regression that meets the normal assumptions, the variances of your parameter estimates can be taken from the variance covariance matrix, $\Sigma$. For example, the variance of the intercept is the first element on the main diagonal, $\Sigma_{11}$. The variance of the slope on $X_1$ is the second element on the main diagonal, $\Sigma_{22}$, and so on.

There are probably many ways to skin a cat, but the standard calculation for the variance covariance matrix uses the residual variance from your model and your design matrix. Then it is:
$$ \rm{VCov(model)} = s^2(X' X)^{-1} $$ Here's a worked example of the calculations with your data and R: 

x = c(1:6);    y = c(18, 14, 15, 12, 7, 6);    m = lm(y ~ x)
summary(m)
# Coefficients:
#             Estimate Std. Error t value Pr(>|t|)    
# (Intercept)  20.4000     1.4119   14.45 0.000133 ***
# x            -2.4000     0.3625   -6.62 0.002700 ** 
# 
# Residual standard error: 1.517 on 4 degrees of freedom
s = summary(m)$sigma;  s  # [1] 1.516575
dm = model.matrix(m);  dm
#   (Intercept) x
# 1           1 1
# 2           1 2
# 3           1 3
# 4           1 4
# 5           1 5
# 6           1 6
s^2*solve(t(dm)%*%dm)
#             (Intercept)          x
# (Intercept)    1.993333 -0.4600000
# x             -0.460000  0.1314286
vcov(m)  # you can see that this is the same as the manual calculation above
#             (Intercept)          x
# (Intercept)    1.993333 -0.4600000
# x             -0.460000  0.1314286
sqrt(diag(vcov(m)))  # these are the same standard errors as the summary output
# (Intercept)           x 
#   1.4118546   0.3625308
Related Question