data analysisleast squaresregressionstatistics
Given a set of data with 11 observations of two variables (response and predictor), I've been asked to "calculate the fitted values $\hat y_i = \hatα + \hatβx'_i $ and residuals $e_i = y_i − \hat y_i$ by hand".
What is the question asking me to do here? I have thus far estimated the regression line for the data in the form $\hat y_i = \hatα + \hatβx'_i $ by calculating the coefficients α & β, but I presume this doesn't answer the original question alone. Where do I go from here? Thanks.
(From the discussion on MO) I guess that the subscript $N$ in $\beta_N$ is a lapse.
It is well know that, conditionally on $X_N$, $\hat \beta_N$ is unbiased and $\mathrm{Cov}(\hat \beta_N) = \sigma^2 \big(X_N^T X_N\big)^{-1}$.
Now in order to have some convergence, one naturally needs additional (to the usual Gauss–Markov) assumptions. Say, if $x_i$ are iid and square integrable, then $$ X_N^T X_N = \sum_{i=1}^N |x_i|^2 \sim n\,\mathrm{E}[|x_1|^2], n\to\infty, $$ almost surely. As a result, $\hat \beta_n \overset{\mathrm P}\longrightarrow \beta$, $n\to\infty$, moreover, $\sqrt{n} |\hat\beta_n - \beta|$ is bounded in probability. (So the convergence rate is loosely $1/\sqrt{n}$.)
What you needs to use to get between (1) and (2) is
which works so long as $\hat\beta_{1} \not =0$
But this is not quite the real issue. Here is an illustration of trying to fit a straight line to something which is actually a parabola plus some noise, using R:
set.seed(2020)
truebeta0 <- 4
truebeta1 <- 7
truebeta2 <- 2
sdnoise <- 3
X <- 1:10
Xsq <- X^2
Y <- truebeta0 + truebeta1 * X + truebeta2 * Xsq + rnorm(10,0,sdnoise)
fit1 <- lm(Y ~ X + Xsq)
fit2 <- lm(Y ~ X)
plot(Y ~ X)
points(fit1$fitted.values ~ X, type="l", col = "blue")
points(fit2$fitted.values ~ X, type="l", col = "red")
to give
and you can see the blue fitted parabola is a good fit with small apparently random residuals, while the red fitted straight line is not so good, with the residuals larger and those in the middle the opposite sign to those at the extremes. If you look at the residuals for the red line, they are indeed visually a parabola plus some noise:
plot(fit2$residuals ~ fit2$fitted.values, type="p", col = "red")
abline(h=0)
Best Answer
If you have calculated $\hat{\alpha}$ and $\hat{\beta}$ you can compute the 11 values of $\hat{y_i}$ by plugging in the 11 values of $x_i$.
Compare the value predicted by the regression, $\hat{y_i}$, and the actual value it should be $y_i$.
Their difference is the residual.