Solved – Standard error of residuals and standard error of regression

least squaresmultiple regressionregressionresidualsstandard error

We know that in simple linear regression the variance of the regression error, $\sigma^2$, is estimated by $\frac {\sum_{i=1}^{n} (y_i – \hat y)^2} {n-2}$, i.e., the Mean Squared Error of the errors. But to standardize the residuals it is said to use the "standard error" of the residuals. Is this the exact same thing as that formula I just wrote?

Or is it $\frac{\sum_{i = 1}^{n} (\epsilon_i – \bar \epsilon)^2}{n-1}$, assuming $\epsilon_i$ is the $i$th residual?. This makes more sense to me. Are the two somehow equivalent?

Best Answer

The standardized residual is the residual divided by its standard deviation. Standardizing residual is a method for transforming data so that its mean is zero and standard deviation is one. If the distribution of the residuals is approximately normal, then $95\%$ of the standardized residuals should fall between $-2$ and $+2$. If many of the residuals fall outside of $+2$ or $ā€“2$, then they could be considered unusual. However, about $5\%$ of the residuals could fall outside of this region due to chance. Consider the regression equation $\hat y_i = \beta_0 + \beta_1x_i + \epsilon_i$ and to compute standardized residuals,

  1. Find the mean of residual, $\bar \epsilon = \frac{\sum_{i=1}^{n} \epsilon_i}{n}$
  2. Calculate the standard deviation of the series, $SD_\epsilon = \sqrt \frac{\sum_{i = 1}^{n} (\epsilon_i - \bar \epsilon)^2}{n} $
  3. Find standardized residual, $s_{\epsilon_i} = \frac{\epsilon_i- \bar \epsilon}{SD_\epsilon}$
Related Question