Solved – F-test on two regression models

f-testmodel comparisonregression

How to compare two simple regression models? Say I have two regression models:

  • Model 1: $Y= c+X_1+X_2$
  • Model 2: $Y= c+X_3+X_4$

How to test null hypothesis that both of these models have equal explanatory power? $R^2s$ I get are pretty close (.044 & .042). Is there any way to do an F-test if models are not nested?

Best Answer

An information criteria such as AIC, AICc or BIC can compare two or more non-nested models as long as they are fit to the exact same response values. Most statistical packages will provide you AIC automatically or it can be called up easily (eg in R by AIC()). They will not, however, check to make sure that the response values are the same, so you have to make sure that NAs among your predictors are not creating small differences in your vector of responses. The R package bbmle has a handy AICtab function for comparing multiple models and can calculate AICc easily.

Related Question