Elastic Net in R – Understanding Lambda in Penalized Regression Models

elastic netr

In the language R there is the possibility to estimate the best $\lambda$ for an elastic net model via a cross-validation, having set the value of $\alpha$. However, being $\alpha$ defined as $\lambda_2/(\lambda_1+\lambda_2)$, where $\lambda_1$ and $\lambda_2$ are the coefficients of the penalizations for the ridge and lasso terms, how is $\lambda$ defined?

Best Answer

You're confused; $\alpha$ and $\lambda$ are totally different.

$\alpha$ sets the degree of mixing between ridge regression and lasso: when $\alpha = 0$, the elastic net does the former, and when $\alpha = 1$, it does the latter. Values of $\alpha$ between those extremes will give a result that is a blend of the two.

Meanwhile, $\lambda$ is the shrinkage parameter: when $\lambda = 0$, no shrinkage is performed, and as $\lambda$ increases, the coefficients are shrunk ever more strongly. This happens regardless of the value of $\alpha$.