Solved – The difference between with or without intercept model in logistic regression

interceptinterpretationlogisticmodelregression

I like to understand the difference between with or without intercept model in logistic regression

Is there any difference between them except that with the intercept the coefficients regard the log(odds ratio) relative to the baseline group and without the intercept they regard the log(odds)? from what I have seen the coefficients are the same in both cases but the significance is not always the same and don't understand why is that.. in addition, in which cases will it be correct to use a model with no intercept?

This is my model: glm(NeverReturn ~ factor(Network) * TotalPrice , family = binomial) and I'm not sure about leave the intercept out or not because at the "real word" the total price cant be under 50 any way,but then the probability will be 1 not 0 so I'm confused.

Best Answer

It will almost never be meaningful to use the no intercept model in logistic regression. The intercept parameter $\beta_0$ is modelling the marginal distribution of the response $Y$, so using $\beta_0=0$ is tantamont to assuming that $P(Y=1)=0.5$, marginally. Do you really know that? If that is untrue, you cannot trust any inference from the no intercept model.

Related Solutions

Regression – Understanding Strange Standard Errors from glm() in R

Your coefficients, even when they share common names, are not the same, i.e. their interpretation is different.

In the first model, the effect of religionChristianity is a variation in the outcome wrt the baseline (religionBuddhism), a relative variation. In the second model the effect of religionChristianity is an absolute variation.

The effects are numerically equal, $-2.8718056+0.4934891=-2.378317+5e-07$, but in the first case the effect is a sum of two effects, i.e. you should compare the joint significance of Intercept and religionChristianity in the first model with the significance of religionChristianity in the second one. You should compare a joint confidence interval (first model) with a simple one (second model).

The simple CI for religionChristianity is:

> confint(my.fit.without.intercept)
Waiting for profiling to be done...
                         2.5 %    97.5 %
...
religionChristianity -2.390467 -2.366207

There are several ways to compute joint intervals. Using arm:

> library(arm)
> n.sims <- 1000
> sim.i <- sim(my.fit, n.sims)
> intercept.plus.christianity <- sim.i@coef[,1] + sim.i@coef[,2]
> quantile(intercept.plus.christianity, c(0.025, 0.975))
     2.5%     97.5% 
-2.390826 -2.366828

Can you see any significant (relevant) difference?

R – Understanding 0 Intercept in Logistic Regression in R

The issue is not specific to a GLM. It's an issue of treatment contrasts.

You should also look at the model with intercept:

set.seed(42)
y <- as.factor(sample(rep(1:2), 30, T))
x <- as.factor(sample(rep(1:2), 30, T))
z <- as.factor(sample(rep(1:2), 30, T))

fit0 <- glm(y ~ z + x, binomial)
predict(fit0, newdata=data.frame(z=factor(2), x=factor(1)))
coef(fit0)
#(Intercept)          z2          x2 
# -0.1151303   0.3228803   1.0588217 
predict(fit0, newdata=data.frame(z=factor(2), x=factor(1)))
#      1 
#0.20775

Here the intercept represents the group x1/z1 and the other group means are calculated by adding the coefficients of z2 and/or x2.

fit1 <- glm(y ~ z + x - 1, binomial)
coef(fit1)
#        z1         z2         x2 
#-0.1151303  0.2077500  1.0588217 
predict(fit1, newdata=data.frame(z=factor(2), x=factor(1)))
#      1 
#0.20775

Here the coefficient of z1 represents the group x1/z1 which is the same as the intercept in fit0. However, the coefficient of z2 represents the group x1/z2 instead of the difference between the group means. Note that 0.208 = -0.115 + 0.323. The x2/* group means are calculated by adding the x2 coefficient to the x1/* group means.

It should now be easy to understand why order matters here.

Best Answer

Related Solutions

Regression – Understanding Strange Standard Errors from glm() in R

R – Understanding 0 Intercept in Logistic Regression in R

Related Question