Recap of required theory
Consider the following regression:
$$y_i = \alpha + \beta x_i + u_i \tag{1}$$
where $y_i$ are iid and $x_i$ are deterministic (i.e. fixed).
We know that the OLS estimator $\hat{\beta}$ is:
$$ \hat{\beta} = \frac{\sum_{i=1}^n (x_i – \bar{x})(y_i – \bar{y})}{\sum_{i=1}^n (x_i – \bar{x})^2}$$
Assume that the Gauss-Markov conditions are fulfilled. Then, the OLS estimator has the minimum variance of any linear (in $y_i$) unbiased estimator.
Setup for my problem
Consider the following class of linear estimators:
$$ \tilde{\beta} = \frac{\sum_{i=1}^n a_i y_i}{\sum_{i=1}^n a_i x_i} \text{ where } \sum_{i=1}^n a_i = 0 \tag{2}$$
Notice that if we set $a_i = (x_i – \bar{x})$, we get the OLS estimator.
This class of estimators is unbiased; we can show this by substituting (1) into (2), leading to:
$$ \tilde{\beta} = \beta + \frac{\sum_{i=1}^n a_i u_i}{\sum_{i=1}^n a_i x_i} \tag{3}$$
Given that the error terms $u_i$ are exogenous, taking expectations through (3) completes the proof that this class of estimators is indeed unbiased.
Why have I bothered to show that this class of estimators is unbiased? Because if it is, then by the Gauss-Markov theorem:
Setting $a_i = (x_i – \bar{x})$ minimizes the variance of $\tilde{\beta}$.
This is the result I want to prove.
My problem: Minimizing the variance of $\tilde{\beta}$
Assuming homoskedasticity i.e $V(u_i) = \sigma^2$, we can use (3) to show that:
$$ V(\tilde{\beta}) = \frac{\sigma^2 \sum_{i=1}^n a_i^2}{\left[\sum_{i=1}^n a_i x_i\right]^2} \tag{4}$$
This is a one-variable optimization problem, since $x_i$ are fixed. Differentiating wrt $a_i$ using the quotient rule yields the first order condition:
$$ \frac{dV}{da} = \frac{2\sigma^2\left[\sum_{i=1}^n a_i x_i\right]^2\left[\sum_{i=1}^n a_i\right] – 2\sigma^2\left[\sum_{i=1}^n a_i x_i^2\right]\left[\sum_{i=1}^n a_i^2\right] }{\left[\sum_{i=1}^n a_i x_i\right]^4} = 0 \tag{5}$$
This simplifies down to:
$$ \left[\sum_{i=1}^n a_i x_i\right]^2\left[\sum_{i=1}^n a_i\right] = \left[\sum_{i=1}^n a_i x_i^2\right]\left[\sum_{i=1}^n a_i^2\right] \tag{6} $$
Using that $\sum_{i=1}^n a_i = 0$, this becomes:
$$\left[\sum_{i=1}^n a_i x_i^2\right]\left[\sum_{i=1}^n a_i^2\right] = 0 \tag{7}$$
…and this is where I get stuck.
How can I solve (7) to show that:
$$a_i = x_i – \bar{x}$$
I realize that this is simply a question of algebraic manipulation (I think I've derived everything up to (7) correctly); I just can't see how to simplify it.
Best Answer
Because $\sum_{i=1}^n a_i = 0$, $\sum_{i=1}^n a_i x_i =\sum_{i=1}^n a_i x_i -\bar x \sum_{i=1}^n a_i = \sum_{i=1}^n a_i (x_i-\bar x)$. So (4) can be written as $$V(\tilde{\beta}) = \frac{\sigma^2 \sum_{i=1}^n a_i^2}{\left[\sum_{i=1}^n a_i x_i\right]^2}=\frac{\sigma^2 \sum_{i=1}^n a_i^2}{\left[\sum_{i=1}^n a_i (x_i - \bar x)\right]^2}$$
According to Cauchy-Schwarz Inequality, we have $$\left[\sum_{i=1}^n a_i (x_i - \bar x)\right]^2 \le \sum_{i=1}^n (x_i-\bar x)^2\sum_{i=1}^n a_i^2$$ Divided by $\left[\sum_{i=1}^n a_i (x_i - \bar x)\right]^2 \sum_{i=1}^n (x_i-\bar x)^2$ on both sides, we get: $$\frac{\sum_{i=1}^n a_i^2}{\left[\sum_{i=1}^n a_i (x_i-\bar x)\right]^2} \ge \frac 1 {\sum_{i=1}^n (x_i-\bar x)^2} = \frac {\sum_{i=1}^n (x_i-\bar x)^2}{\left(\sum_{i=1}^n (x_i-\bar x)^2\right)^2}$$ So $V(\hat \beta) \le V(\tilde \beta)$