Weighted linear regression coefficient without intercept’s closed-form solution differs from R’s lm(.., weights=)

rweighted-regression

I am working through Harold Larson's Introduction to Probability Theory and Statistical Inference. In Chapter 9, Example 9.1.4 he discusses the simple linear regression model but where we assume the variance increases linearly with $x_i$. For the model $y = b x$ we suppose $E[Y_i] = b x_i$, where $x_i$ is the true weight, the $Y_i$'s are uncorrelated, but now $Var[Y_i] = \sigma^2 x_i$. The best linear unbiased estimate for $b$ is the value which minimizes

$$ Q = \sum \frac{ (y_i – b x_i)^2}{x_i} $$

Gives

\begin{aligned}
\frac{dQ}{db} = -2 \sum \frac{x_i(y_i – b x_i)}{x_i} \\
= -2 \sum y_i + 2b \sum x_i
\end{aligned}

and we have

$$\hat b = \frac{\sum y_i}{\sum x_i} = \frac{\frac{1}{n} \sum y_i}{\frac{1}{n} \sum x_i} = \frac{\bar y}{\bar x}$$

Then in Exercise 9.1.5 we are asked to estimate $b$ for the model $y = b x$ using the data

x <- c(10, 20, 30, 40, 50, 60, 70, 80, 90, 100)
y <- c(9.03, 19.31, 28.94, 39.84, 49.12, 59.05, 68.69, 78.32, 87.79, 97.70)

According to the above derivation, the estimate for $b$ should be bhat <- mean(y) / mean(x) which is .9778. This is the also the answer given in the solution to the exercise at the back of the book.

However, if I use R's lm function with no intercept and the weights parameter:

lm(y ~ 0 + x, weights = x)

I get the following output from the summary:

Call:
lm(formula = y ~ 0 + x, weights = x)

Weighted Residuals:
    Min      1Q  Median      3Q     Max 
-2.5418 -2.0505 -0.3709  1.6098  4.4471 

Coefficients:
   Estimate Std. Error t value Pr(>|t|)    
x 0.978421   0.001388   704.7   <2e-16 ***

The coefficient for x is estimated as .978421, which is slightly different from the closed form value of .9778.

Can anyone help me understand what R does differently, or whether I am making a mistake?

Best Answer

help("lm") explains

weighted least squares is used with weights weights (that is, minimizing sum(w*e^2))

According to your first equation, this means you need to specify w = 1/x.

all.equal(unname(coef(lm(y ~ 0 + x, weights = 1/x))), 
          mean(y)/mean(x))
#[1] TRUE
Related Question