Solved – How is it possible that R square increases and the F value decreases

multiple regressionr-squared

In a multiple regression I start with two independent variables and in the following two blocks I am adding more predictors every time. The R Square is increasing with each addition of predictors, but the F-value is decreasing. How is this possible or what is done wrong perhaps?

Best Answer

Yes, this is possible, although it should be rare in practice. There are two parts:

  1. If you add variables to a model, $R^2$ has to either go up or stay the same. It cannot go down. Moreover, it can only stay the same if the variable added is perfectly uncorrelated with the response, not just in the population, but in your sample. As a result, $R^2$ will essentially always go up.
  2. The $F$-statistic will typically increase as well, although it may not increase faster than the df associated with the statistics. In the latter case, $F$ would increase, but become less significant. In order for the $F$-statistic to go down, the added variables have to be so little correlated with the response that you lose more error degrees of freedom than you lose actual sums of squared errors. In this way, your mean square error (their quotient) will increase. With a larger denominator for the $F$-statistic, it can go down. It may help you to read my answer here: How can adding a 2nd IV make the 1st IV significant?
Related Question