This seems to be a basic issue, but I just realized that I actually don't know how to test equality of coefficients from two different regressions. Can anyone shed some light on this?
More formally, suppose I ran the following two regressions:
$$
y_1 = X_1\beta_1 + \epsilon_1
$$
and
$$
y_2 = X_2\beta_2 + \epsilon_2
$$
where $X_i$ refers to the design matrix of regression $i$, and $\beta_i$ to the vector of coefficients in regression $i$. Note that $X_1$ and $X_2$ are potentially very different, with different dimensions etc. I am interested in for instance whether or not $\hat\beta_{11} \neq \hat\beta_{21}$.
If these came from the same regression, this would be trivial. But since they come from different ones, I am not quite sure how to do it. Does anyone have an idea or can give me some pointers?
My problem in detail: My first intuition was to look at the confidence intervals, and if they overlap, then I would say they are essentially the same. This procedure does not come with the correct size of the test, though (i.e. each individual confidence interval has $\alpha=0.05$, say, but looking at them jointly will not have the same probability). My "second" intuition was to conduct a normal t-test. That is, take
$$
\frac{\beta_{11}-\beta_{21}}{sd(\beta_{11})}
$$
where $\beta_{21}$ is taken as the value of my null hypothesis. This does not take into account the estimation uncertainty of $\beta_{21}$, though, and the answer may depend on the order of the regressions (which one I call 1 and 2).
My third idea was to do it as in a standard test for equality of two coefficients from the same regression, that is take
$$
\frac{\beta_{11}-\beta_{21}}{sd(\beta_{11}-\beta_{21})}
$$
The complication arises due to the fact that both come from different regressions. Note that
$$
Var(\beta_{11}-\beta_{21}) = Var(\beta_{11}) + Var(\beta_{21}) -2 Cov(\beta_{11},\beta_{21})
$$
but since they are from different regressions, how would I get $Cov(\beta_{11},\beta_{21})$?
This led me to ask this question here. This must be a standard procedure / standard test, but I cound not find anything that was sufficiently similar to this problem. So, if anyone can point me to the correct procedure, I would be very grateful!
Best Answer
Although this isn't a common analysis, it really is one of interest. The accepted answer fits the way you asked your question, but I'm going to provide another reasonably well accepted technique that may or may not be equivalent (I'll leave it to better minds to comment on that).
This approach is to use the following Z test:
$Z = \frac{\beta_1-\beta_2}{\sqrt{(SE\beta_1)^2+(SE\beta_2)^2}}$
Where $SE\beta$ is the standard error of $\beta$.
This equation is provided by Clogg, C. C., Petkova, E., & Haritou, A. (1995). Statistical methods for comparing regression coefficients between models. American Journal of Sociology, 100(5), 1261-1293. and is cited by Paternoster, R., Brame, R., Mazerolle, P., & Piquero, A. (1998). Using the correct statistical test for equality of regression coefficients. Criminology, 36(4), 859-866. equation 4, which is available free of a paywall. I've adapted Peternoster's formula to use $\beta$ rather than $b$ because it is possible that you might be interested in different DVs for some awful reason and my memory of Clogg et al. was that their formula used $\beta$. I also remember cross checking this formula against Cohen, Cohen, West, and Aiken, and the root of the same thinking can be found there in the confidence interval of differences between coefficients, equation 2.8.6, pg 46-47.