Solved – Two intercept coefficients in glmnet output

glmnetinterceptr

I am not sure I understand why glmnet returns 2 intercepts in its result.
If model matrix already has intercept column, why do we need another intercept for the model?

> mm = model.matrix(...)
> colnames(mm)
 [1] "(Intercept)"  "v1" "v2"....

> model.fit = glmnet(mm, resp, standardize=FALSE, family='binomial', nlambda = 10)
> coef(model.fit, s= 1e-05)
  48 x 1 sparse Matrix of class "dgCMatrix"                        1
  (Intercept)        5.9487109234
  (Intercept)        .           
  v1        0.2580378293
  v2       -0.4693849121
  .....

Best Answer

You need to drop the intercept (the vector of 1's) from the mm matrix since the glmnet package automatically demeans the data and reports the intercept term by default.

Alternatively, you can use the intercept parameter to glmnet (TRUE by default).

Related Solutions

Solved – Predict Coefficients with glmnet()

If you call

ridge.fit1 <- glmnet(x,y, alpha = 0)

the lasso is fit for a grid of 100 values of the regularization parameter lambda. If you call the predict function for a value of lambda that is not a grid point itself, the coefficients are extrapolated.

If you call

ridge.fitt <- coef(glmnet(x, y, alpha = 0, lambda = 3))

the lasso is fit only for a lambda value of 3, and the exact coefficients for this fit are returned. You can make the predict function refit the model using the given lambda value and return exact coefficients by adding the argument exact=TRUE.

Solved – glmnet cross-validation without intercept

glmnet optimizes the following loss function:

$\sum_{i=1}^n (\hat{Y}_i-Y_i)^2 + \lambda\left(\frac{(1-\alpha)}{2}||\beta||_2^2 + \alpha ||\beta||_1 \right)$

The residual sums of squares is on the left, as typical with regression, and the penalization for the coefficients is on the right. $\alpha$ defaults to 1, which gives the LASSO penalty.

Now, if you don't fit an intercept, the term on the left will be very large (if $E(Y)$ is large). The model will try to account for that, but it will require larger coefficient values to account for the intercept. It may be the case (and I'm guessing here) that you have $E(Y)$ large and one of your variables is fairly constant. In that case, that variable will get a large coefficient (as it helps to reduce the SSR), but other variables increase the penalization to much and hence there coefficients are zero.

Maybe you could supply your own lambda sequence to the function, something like

lambda=10^seq(1,-4,-.5)

If $\lambda$ is small enough, you should get more non-zero coefficients in the model without an intercept as well.

Note: I don't think this problem has anything to do with the fact you're using cv.glmnet. You should see the same thing if you just use glmnet.

Best Answer

Related Solutions

Solved – Predict Coefficients with glmnet()

Solved – glmnet cross-validation without intercept

Related Question