Solved – Obtaining residual variance from glmnet

glmnetr

I am working with glmnet and I'd like to see how we can obtain a residual variance for a given penalty parameter in prediction. I use glmnet's QuickStartExample example (here) and focus on L1 penalty with a given lambda. My code is as follows:

require(glmnet)
data(QuickStartExample)
fit <- glmnet(x, y, intercept = F, standardize = F, thresh = 1e-20, alpha = 1)

Following, I extracted coefficient vector using predict function to get the exact solution for 25th lambda as follows:

beta_glmnet = as.matrix(predict(fit, s = fit$lambda[25]/(2*dim(x)[1]), type = "coefficients", exact = T)[-1,])

My question is how we can obtain an estimate (preferably an unbiased one) of residual variance for the above model. My initial thought was fit$dev.ratio[25]/fit$df[25].

Any ideas?

Best Answer

In my understanding it is in deviance function of glmnet

data.table(l=fit$lambda, d=deviance(fit))[abs(l-reg.lambda) == min(abs(l-reg.lambda))]$d[1]

Related Solutions

GLMNET – Determining Variable Importance in Logistic Regression

As far as I know glmnet does not calculate the standard errors of regression coefficients (since it fits model parameters using cyclic coordinate descent). So, if you need standardized regression coefficients, you will need to use some other method (e.g. glm)

Having said that, if the explanatory variables are standardized before the fit and glmnet is called with "standardize=FALSE", then the less important coefficients will be smaller than the more important ones - so you could rank them just by their magnitude. This becomes even more pronounced with non-trivial amount shrinkage (i.e. non-zero lambda)

Hope this helps..

Solved – Extracting regularization path from glmnet

glmnet provides a full coefficient path. Try the following example out for yourself:

# code partially scavenged from the glmnet docs because I'm lazy
library(glmnet)

set.seed(123)

#binomial
x=matrix(rnorm(100*20),100,20)
g2=sample(1:2,100,replace=TRUE)
fit2=glmnet(x,g2,family="binomial")

plot(fit2) # plot coefficient paths

summary(fit2) # NB: 63 lambda values tested

dim(fit2$beta) # 20 by 63, as we'd expect

The number of lambdas tested is contingent on the random seed and properties in the data. To illustrate this, change the above example such that you are starting with set.seed(111) and you'll end up with 55 lambdas tested instead of 63. It's possible that lasso is converging extremely quickly for you and glmnet isn't even bothering to give you more than one lambda. I doubt that's the case, but it's probably possible.

If you are specifying lambda in your call to glmnet, then you won't get a coefficient path since the path is only meaningful over varying lambdas. If this doesn't answer your question, try to provide a minimal working example of your code.

Best Answer

Related Solutions

GLMNET – Determining Variable Importance in Logistic Regression

Solved – Extracting regularization path from glmnet

Related Question