Following is a note from page 1 in: Matrix OLS
Make sure that you are always careful about distinguishing between
disturbances ($\epsilon$) that refer to things that cannot be observed
and residuals ($e$) that can be observed. It is important to remember
that $\epsilon$ $\not =$ $e$.
What is the difference between $\epsilon$ and $e$? Given that $\epsilon$ is considered unobserved, in what sense are we able to use this value for OLS?
Best Answer
The disturbance, or error term, represents factors other than x that affect y. Understanding the error term is useful in OLS because it is the subject of many of our important assumptions.
Since we do not observe errors, we resort to looking at residuals, which can give us an idea about the underlying errors. While errors are unobservable, residuals are observable: we can calculate residuals; that is, we can calculate the difference between each of our y values and their corresponding fitted values that lie on the regression line.
As for the aforementioned assumptions, two important ones of the Classical Linear Model Assumptions are (1) Normality of Error Terms and (2) Homoscedasticity.
Assumption (1): We assume that the unobserved factors are normally distributed around the population regression function. This assumption is for the purpose of statistical inference, but it is not crucial: this assumption can be replaced by the assumption of a large sample size.
Assumption (2): We assume that the error $\epsilon$ has the same variance given any value of the explanatory variables. This assumption is critical in OLS. If the model exhibits heteroscedasticity, then our t statistics do not have t distributions, and our f statistics do not have f distributions; and thus, our statistical inference is no longer reliable. This assumption CANNOT be replaced by the assumption of a large sample size. Though, there are methods for dealing with heteroscedasticity.
There are also other assumptions from the Classical Linear Model Assumptions that rely on our understanding the error term, but those are beyond the scope of this post.
One last note: we have assumed use of cross-sectional data.