The assumed regression model is $E(Y_i|x)=\gamma+\beta(x-\bar{x})$ and $Var(Y|x)=\sigma^2$.
So I have:
$E(RSS)=
E(\sum\limits_{i=1}^n (y_i-\hat{y_i})^2)=
E(\sum\limits_{i=1}^n (y_i-\hat{\gamma}-\hat{\beta}(x_i-\bar{x}))^2)=
E(\sum\limits_{i=1}^n (y_i^2-2y_i\hat{\gamma} + \hat{\gamma}^2 – 2\hat{\beta}(x_i-\bar{x})y_i + 2\hat{\beta}(x_i-\bar{x})\hat{\gamma} + \hat{\beta}^2(x_i-\bar{x})^2)=
\sum\limits_{i=1}^n (E(y_i^2) – 2E(y_i\hat{\gamma})+
E(\hat{\gamma}^2) – 2(x_i-\bar{x})E(y_i\hat{\beta})+
2(x_i-\bar{x})E(\hat{\gamma}\hat{\beta})+
(x_i-\bar{x})^2E(\hat{\beta}^2)).$
But here I'm stuck.
For $E(y_i^2), E(\hat{\gamma})$ and $E(\hat{\beta}^2)$
I can use the definition $E(X^2)= Var(X)+(E(X))^2$.
But what should I use for the others?
Clearly, $y_i$ and $\hat{\gamma}$ are dependent, then do I need to find their covariance to take the expectation?
$\hat{\gamma}$ and $\hat{\beta}$ are uncorrelated by definition so I would know what to with them.
However, I am not sure about $y_i$ and $\hat{\beta}$, are they correlated?
Best Answer
From Rice - Mathematical Statistics and Data Analysis, using matrix notation.
Consider the sum of squared residuals for the general linear regression problem $||\mathbf{Y-HY}||^2$, where $\mathbf{H=X(X^TX)^{-1}X}$, then:
$$ \mathbb{E}||\mathbf{Y-HY}||^2 = \mathbb{E}(\mathbf{Y^T(I-H)Y}) = [\mathbb{E}(\mathbf{Y})]^T(\mathbf{I-H})[\mathbb{E}(Y)] + \sigma^2 tr(\mathbf{I-H}). $$
As $\mathbb{E}(\mathbf{Y})=\mathbf{X\beta}$, we get $(\mathbf{I-H})[\mathbb{E}(Y)] = 0$. Furthermore, $tr(\mathbf{I-H})= tr(\mathbf{I}) - tr(\mathbf{H})= n-p$.
Thus, $$\mathbb{E}||\mathbf{Y-HY}||^2 = (n-p) \sigma^2$$
For your two parameter model, we then have:
$$ \mathbb{E}(RSS) = (n-2)\sigma^2 \> \square $$