Weighted Regression – Definition of Weights in R lm Function vs. Mathematical Expression

least squaresrregressionweighted-dataweighted-regression

Could anyone tell me why I am getting different results from R weighted least squares and manual solution by matrix operation?

Specifically, I am trying to manually solve $\mathbf W \mathbf A\mathbf x=\mathbf W \mathbf b$, where $\mathbf W$ is the diagonal matrix on weights, $\mathbf A$ is the data matrix, $\mathbf b$ is the response vector.

I am trying to compare the results with the R lm function using the weights argument.

enter image description here

Best Answer

As you can see from the mathematical expressions for your calculations, you are obtaining

$$((WA)^\prime (WA))^{-1} \; ((WA)^\prime (Wb)) = (A^\prime W^2 A)^{-1} (A^\prime W^2 b).$$

Evidently your weights are $W^2$, not $W$. Thus you should be comparing your answer to the output of

> lm(form, mtcars, weights=w^2)
Coefficients:
      wt        hp      disp  
14.12980   0.08391  -0.16446 

The agreement is perfect (to within floating point error--internally, R uses a numerically more stable algorithm.)