Solved – Glmnet: How to select Lambda and Alpha

elastic netglmnetlassorridge regression

I'd like to pick the optimal lambda and alpha using the Glmnet package. I'm open to all models (Ridge, Lasso, Elastic). I'm assuming some out of sample error/cross validation is the best model selection criteria.

Macro <- read.csv("P:/Earnest/Old/R/Input.csv")
x <- Macro[1:13,3:21]
x <- as.matrix(x)

y <- Macro[1:13,2:2]
y <- as.matrix(y)

t <- Macro[14:14,3:21]
t <- as.matrix(t)

Right now, I'm using the following code. The below code presupposes alpha = .5 (elastic), and that lambda.min is the ideal lambda. 

fit <-glmnet(x, y, alpha = .5, lambda = NULL) 
cv.fit=cv.glmnet(x,y, alpha = .5, lambda = NULL)
min <- cv.fit$lambda.min
predict(fit ,t, s = min)

Questions: How do I know what is the ideal alpha and lambda? What code can I use to test various lambda/alpha combinations, to find the best out of sample error. Is this the right approach and utilization of Glmnet? What questions am I not considering that should be?

The data:
enter image description here

Best Answer

It appears that the default in glmnet is to select lambda from a range of values from min.lambda to max.lambda, then the optimal is selected based on cross validation. The range of values chosen by default is just a linear range (on the log scale) from a the minimum value (like 0, or some value for which we set no features to zero) to the maximum value, (which they set to the smallest value for which the model would set all features to zero).

From the glmnet documentation:

lambda can be provided, but is typically not and the program constructs a sequence. When automatically generated, the λ sequence is determined by lambda.max and lambda.min.ratio. The latter is the ratio of smallest value of the generated λ sequence (say lambda.min) to lambda.max. The program then generates nlambda values linear on the log scale from lambda.max down to lambda.min. lambda.max is not given, but easily computed from the input x and y; it is the smallest value for lambda such that all the coefficients are zero. For alpha=0(ridge) lambda.max would be ∞; hence for this case we pick a value corresponding to a small value for alpha close to zero.)

Related Question