Solved – Logistic predict.glm() function error: factor has new levels

error messagegeneralized linear modellogisticr

I fit a logistic on three numeric continuous variables, followed by a categorical factor [Y, N].

logit2A <- glm(DisclosedDriver ~ VehDrvr_Dif+POL_SEQ_NUM+PRMTOTAL+SAFE_DRVR_PLEDGE_FLG, data = DF, family = "binomial")

Fit looks wonderful.

Coefficients:
                    Estimate Std. Error z value Pr(>|z|)    
(Intercept)           -2.204e+00  2.253e-01  -9.782  < 2e-16 ***
VehDrvr_Dif            2.918e-01  1.026e-01   2.845 0.004440 ** 
POL_SEQ_NUM           -1.893e-01  5.617e-02  -3.370 0.000751 ***
PRMTOTAL               1.109e-04  5.526e-05   2.006 0.044804 *  
SAFE_DRVR_PLEDGE_FLGY -7.220e-01  1.633e-01  -4.422 9.76e-06 ***

So obviously R took the Safe_Drvr_Pledge_Flg categorical factor variable and placed all 'N' values in reference or intercept as opposed to the listed 'Y'.

Now I want to take my fit and calculate the probabilities that my model determines. And here comes the error:

> DF$P_GLM<- predict.glm(logit2A, DF, type="response", se.fit=FALSE)
    Error in model.frame.default(Terms, newdata, na.action = na.action, xlev = object$xlevels) : 
factor SAFE_DRVR_PLEDGE_FLG has new levels

Umm… no it doesn't, because I just fit the model with the exact same data I'm trying to use for the prediction. What's the problem?

Trying to respond to first comment:
Don't know what you mean. I've got 3500 rows of data… It's a logistic regression on 4 continuous variables and one categorical. The categorical has two values, Y or N. My glm fit give the numbers given. I just want to plug it all back in with the predict function and it gives me that error. Here's the categorical variable:

 > DF$SAFE_DRVR_PLEDGE_FLG
 [1] Y Y Y Y Y Y Y N Y Y Y Y Y Y Y Y Y Y Y N Y Y N Y Y N Y Y N Y Y Y Y Y Y Y Y Y Y N Y Y N Y Y Y N Y Y Y Y Y N Y Y Y Y Y Y
 [60] Y Y Y Y N Y Y Y Y Y Y Y Y N Y Y Y N N Y N Y Y Y Y Y N Y Y N Y N N Y Y Y N Y Y Y Y N Y Y Y Y Y N Y N Y N Y Y Y Y Y N Y
 [119] N Y Y Y Y Y Y Y Y N Y Y Y Y Y Y N Y Y Y N Y Y Y N Y Y Y N N Y N N N Y N Y Y Y N N Y Y N Y Y Y Y N N Y Y Y Y N N Y N N
 Levels:  N Y

What do you mean by a working example? The fit works. The probability output of the predict function doesn't…

Best Answer

Using my magic crystal ball to see the output of str(DF)...

Aha! SAFE_DRVR_PLEDGE_FLG has another level, "", and that in that same row, one of the other variables is missing. So it's not using that row to do the fit, but when it tries to construct the data matrix to get the predictions, it can't because the level "" exists in the data set but not in the model.

You should probably fix your data set before fitting the model, but you can also get the predictions to work by not telling predict to use the DF data frame; it will then successfully work as it uses the data matrix from the model. However, this will have fewer predictions than there are rows in DF because those rows that were causing trouble were automatically discarded.

Example, based on @smillig's:

d <- data.frame(w=rnorm(100),
                x=rnorm(100), 
                y=sample(LETTERS[1:2], 100, replace=TRUE),
                z=sample(LETTERS[3:4], 100, replace=TRUE) )
d2 <- rbind(d, data.frame(w=1, x=1,y=NA, z=""))
fm2  <- glm(y ~ w + x + z, data=d2, family=binomial)

predict.glm(fm2, d2, type="response", se.fit=FALSE)
# Error in model.frame.default(Terms, newdata, na.action = na.action, xlev = object$xlevels) : 
# factor 'z' has new level(s) 

predict(fm2, type="response", se.fit=FALSE)
# No error, this works. However, it has length 100 even though d has 101 rows.

Edit:

Now, if you really do want to model all those interaction terms, there are certain approaches you could take. For instance, you can check out this set of lecture notes for strategies for dealing with sparse matrices. However, I think the easiest thing here would be to manually add a handful of interaction terms since it's usually best to keep things as simple as they need to be (but no simpler).

Solved – Missing factor levels after logistic regression glm()

This line in glm() is doing you in:

mf$drop.unused.levels <- TRUE

which is effectively setting the argument of the same name of model.frame(), which results in the behaviour you report.

The obvious solution is to not allow this to happen, to adjust your split sampling algorithm you use to produce your training and test sets. Instead of randomly sampling the rows of the data randomly sample within the levels of the factor.

If you don't want to handle the details yourself, try the caret package and its function createFolds():

## install.packages("caret")
library("caret")

X1 <- factor(rep(1:3, times = c(20, 30, 50))) ## dummy data for illustration
f <- createFolds(X1, k = 5)
f

which gives:

> f <- createFolds(X1, k = 5)
> f
$Fold1
 [1]  5  7 10 20 21 24 29 31 34 42 51 52 59 68 75 76 82 83
[19] 85 94

$Fold2
 [1]  4  9 11 18 22 23 30 38 40 44 55 58 62 66 70 72 80 81
[19] 87 92

$Fold3
 [1]  1 12 14 16 27 37 41 48 49 50 53 60 61 63 64 74 79 88
[19] 89 97

$Fold4
 [1]  3 15 17 19 25 28 32 35 36 43 54 57 67 69 71 73 78 86
[19] 98 99

$Fold5
 [1]   2   6   8  13  26  33  39  45  46  47  56  65  77  84
[15]  90  91  93  95  96 100

The values in f are the indices of the elements of X1 partitioning it into k = 5 groups, with sampling from within the levels of X1 as needed. Then take 1 of these folds at random as the test set.

## number of samples in levels of X1 for each split
> table(X1[-f[[1]]])

 1  2  3 
16 24 40 
> table(X1[-f[[2]]])

 1  2  3 
16 24 40 
> table(X1[-f[[3]]])

 1  2  3 
16 24 40 
> table(X1[-f[[4]]])

 1  2  3 
16 24 40 
> table(X1[-f[[5]]])

 1  2  3 
16 24 40

Do note that this algorithm doesn't guarantee that for small sample sizes that the stratified sampling will always work (i.e you may not be able to escape the missing levels issue in all cases).

Best Answer

Related Solutions

Logistic Regression – Binary Logistic Regression with Interaction Terms

Edit:

Solved – Missing factor levels after logistic regression glm()

Related Question