Let us work with the following structural model: $$y=\mathbf{x_{1i}β}+x_{2i}β_2+\varepsilon_i$$
where $x_{2i}$ is our single endogenous regressor. It turns out that given my instruments and my first stage regression, I can obtain $β_2$ in two different ways. I can use the two stage least squares regression $$y=\mathbf{x_{1i}β}+\hat{x}_{2i}β_2+\varepsilon_i \qquad (*)$$ where $\hat{x}_{2i}$ is our instrument predicted with the first stage regression.
And we can obtain the same estimate for $β_2$ with the following regression
$$y=\mathbf{x_{1i}\beta}+x_{2i}β_2+\delta\hat{v}_i+\varepsilon_i \qquad (**)$$
where $\hat{v}_i$ is the estimated residual of the first stage regression, namely the portion of the original endogenous regressor that is correlated with the original $\varepsilon_i$.
Now I would like to prove algebraically that we can estimate $β_2$ using these two different approaches. I understand why the estimates must be the same. After all, in $(*)$ we are estimating the partial effect of $x_{2i}$ on $y$ controlling for the portion of $x_{2i}$ that was correlated with the error in the structural model above. Is this interpretation correct?
Furthermore I tried to explicit this algebraically. I know I should be able to rewrite $(*)$ as $(**)$ or viceversa. Unfortunately I am not convinced of my result. I started from $(*)$, used the fact that $\hat{v_i}=x_{2i}-\hat{x}_{2i}$ and got the following result
$$y=\mathbf{x_{1i}β}+x_{2i}β_2-β_2\hat{v_i}+\varepsilon_i$$
Now, this result is similar to $(**)$ but I cannot understand it. In particular, how should I decompose $x_{2i}$? What am I missing? Could you give me a hint?
Thank you for your help.
Best Answer
When I am trying to figure out why a result is true, I usually look at it in the simplest possible case. So, let's try a bivariate system. Here is the structural system: \begin{align} y_i &= \beta x_i + \epsilon_i\\ x_i &= \delta z_i + \nu_i \end{align} Here, everything has a zero mean, and $z$ is a valid instrument for $x$. For 2SLS and CF, we run auxilliary regressions like: \begin{align} y_i &= \beta_\text{2SLS} \hat{x}_i + \epsilon_{2i}\\ y_i &= \beta_\text{CF} x_i + \delta \hat{\nu}_i + \epsilon_{3i} \end{align} Now, the 2SLS estimator will be: \begin{align} \hat{\beta}_\text{2SLS} &= \frac{\sum y_i \hat{x}_i}{\sum (\hat{x}_i)^2} \end{align} By the Frisch-Waugh-Lovell Theorem, the CF estimator will be: \begin{align} \hat{\beta}_\text{CF} &= \frac{\sum e_{y|\hat{\nu},i} e_{x|\hat{\nu},i}}{\sum (e_{x|\hat{\nu},i})^2} \end{align} In that expression, $e_{y|\hat{\nu}}$ means the residuals from a regression of $y$ on $\hat{\nu}$.
If you regress $x$ on $\hat{\nu}$, then you get a coefficient of 1 and the residuals from that regression are $\hat{x}$. So, the denominators of the two fractions are the same. Also, this gives you that the second terms in the numerators are the same.
What happens when you regress $y$ on $\hat{\nu}$? Well, the residuals from that regression are: \begin{align} y-\left(\frac{\sum y\hat{\nu}}{\sum(\hat{\nu})^2}\right)\hat{\nu} \end{align}
So, the conclusion follows from the fact that $\sum \hat{\nu}\hat{x}=0$.
It should not be hard to generalize this to the case with an intercept and more right hand side variables.