Variance Estimation – What is the Variance of $s^2$?

consistencyestimationsamplevariance

I am trying to calculate the variance of $s^2=\frac{1}{n-1}\sum (x_i-\bar x)^2$. So what I want to find is $ Var(s^2)$.

I have seen different posts, but many of them seem to make the assumption that the population is normally distributed. I don't know anything about the distribution.

Can anyone help or guide me in the right direction?

Best Answer

The variance of s^2 if the population is non-Normal in case of simple random sampling, is

       Var(s^2) = (1 / n)[μ4 - μ2^2(n - 3) / (n -1)]

If the population is Normal, then μ4 = 3σ^4, and μ2^2 = σ4. So, we get

                  Var(s^2) = 2σ^4 / (n - 1)

Related Solutions

Solved – Variance of slope

For a standard linear regression that meets the normal assumptions, the variances of your parameter estimates can be taken from the variance covariance matrix, $\Sigma$. For example, the variance of the intercept is the first element on the main diagonal, $\Sigma_{11}$. The variance of the slope on $X_1$ is the second element on the main diagonal, $\Sigma_{22}$, and so on.

There are probably many ways to skin a cat, but the standard calculation for the variance covariance matrix uses the residual variance from your model and your design matrix. Then it is:
$$ \rm{VCov(model)} = s^2(X' X)^{-1} $$ Here's a worked example of the calculations with your data and R:

x = c(1:6);    y = c(18, 14, 15, 12, 7, 6);    m = lm(y ~ x)
summary(m)
# Coefficients:
#             Estimate Std. Error t value Pr(>|t|)    
# (Intercept)  20.4000     1.4119   14.45 0.000133 ***
# x            -2.4000     0.3625   -6.62 0.002700 ** 
# 
# Residual standard error: 1.517 on 4 degrees of freedom
s = summary(m)$sigma;  s  # [1] 1.516575
dm = model.matrix(m);  dm
#   (Intercept) x
# 1           1 1
# 2           1 2
# 3           1 3
# 4           1 4
# 5           1 5
# 6           1 6
s^2*solve(t(dm)%*%dm)
#             (Intercept)          x
# (Intercept)    1.993333 -0.4600000
# x             -0.460000  0.1314286
vcov(m)  # you can see that this is the same as the manual calculation above
#             (Intercept)          x
# (Intercept)    1.993333 -0.4600000
# x             -0.460000  0.1314286
sqrt(diag(vcov(m)))  # these are the same standard errors as the summary output
# (Intercept)           x 
#   1.4118546   0.3625308

Maximum Likelihood Estimator – Variance of the $\hat{\sigma}^2$

Do you know the famous result that if $X_1, \ldots, X_n \text{ i.i.d. } \sim N(\mu, \sigma^2)$, then $$\frac{1}{\sigma^2}\sum_{i = 1}^n (X_i - \bar{X})^2 \sim \chi_{n - 1}^2?$$ It is also well-known that the variance of a $\chi_k^2$ random variable is $2k$.

Best Answer

Related Solutions

Solved – Variance of slope

Maximum Likelihood Estimator – Variance of the $\hat{\sigma}^2$

Related Question