[Math] Matrix form for Weighted Least Squares

regressionstatistics

If we have the following weighted least-squares regression, with

$\hat{\beta} = (X'WX)^{-1}X'WY$

How can we express the squared errors, MSE and the fitted values in matrix form?

These are the OLS equivalent:

$Squared Error_i = (\hat{y}_i – y_i )^2$

$Mean Squared Error(MSE) = \Sigma(\hat{y}_i – y_i)^2/(N-p)$

$Fitted Value_i = \hat{y}_i= \hat{\beta} * x_i$

Thank you!

Best Answer

One approach is to consider transformed variables. If you define e.g. $$ X^*=W^{1/2}X\\ Y^*=W^{1/2}Y $$ and apply this transformation, you can write your estimator as $$ \hat{\beta}=(X'WX)^{-1}X'WY=({X^*}'X^*)^{-1}{X^*}'Y^* $$ which is regular OLS, but it is applied to a transformed regression. Can you see what such a transformation means in terms of the scalar quantities you are looking for?