Solved – Calculating $R^2$ for Elastic Net

elastic netmodel-evaluationr-squared

I am trying to do variable selection using elastic net (Matlab Lasso function with alpha of 0.5). I have 75 predictors in total (some are correlated with each other, hence using elastic net instead of lasso), and I would like to get a subset of them, which are good predictors for my outcome.

So my question is: How can I calculate something like $R^2$ that show how much of my outcome is explained by these selected variables?

If I use the selected variables in a multiple linear regression model, is the $R^2$ gonna be valid, since my variables are correlated?
Can I calculate cross-validated $R^2$ (using leave-one-out) to get a more accurate $R^2$?
Is there any other way than calculating $R^2$ that I show my variable selection method predicts well?

Best Answer

Just use the regular $R^2$, i.e. the squared correlation between the fitted and the actual values. Whether the model was fit by OLS or by penalized OLS (such as the elastic net), it will still reflect the proportion of variance explained.

Be aware, however, that model diagnostics and performance measures (such as $R^2$) applied after model selection may (and will) be overly optimistic if the model is evaluated on the same data that was used for model building (e.g. variable selection).

Apart from the warning above, correlated variables are not a problem for $R^2$.
If you were to predict the left-out fold in $K$-fold cross validation and base the $R^2$ on prediction accuracy, then it is not a very useful measure because $R^2$ ignores prediction bias and only accounts for prediction variance. But I am not sure I understand what you mean by cross-validated $R^2$.
Split your data into training, validation and test subsets. Train your models on the training data and use validation data to pick the best-performing model. Re-estimate your selected model on test+validation data. Then assess the performance of the re-estimated model on the test data. Use mean squared error instead of $R^2$ since to properly account for any prediction bias extra to prediction variance.

Related Solutions

Solved – Stepwise regression vs. elastic net

Your question has an implicit assumption that $R^2$ is a good measure of the quality of the fit and is appropriate for comparing between models. I think that your background information provides evidence that $R^2$ is not a good tool for what you are trying to do. After all, you can increase $R^2$ by adding nonsense variables to your model.

Did you take the variables that were found using the elastic net and refit a new regression model using those variables rather than use the estimates from the elasticnet fit? That is kind of like entering your data into a nice statistical software program and using it to round the data and print it out so you can calculate the mean using an abacus.

If you want the fewest predictors possible (and still get a reasonable fit) then lasso methods will tend to result in fewer predictors than elasticnet methods. The advantage of the elasticnet method is not in finding the fewest variables, but in finding a good model that takes advantage of the information in the variables and avoids the bias that you get with stepwise models.

A better comparison would be how well they predict a new set of observations, or maybe a press statistic or cross-validation.

Solved – Elastic net arbitrary alpha selection

The answer to a similar question here advises to follow the glmnet vignette (assuming you're using R):

foldid=sample(1:10,size=length(y),replace=TRUE)
cv1=cv.glmnet(x,y,foldid=foldid,alpha=1)
cv.5=cv.glmnet(x,y,foldid=foldid,alpha=.5)
cv0=cv.glmnet(x,y,foldid=foldid,alpha=0)

Keep the foldid fixed and assess a grid of $\alpha$ values using cross-validation for $\lambda$.

Best Answer

Related Solutions

Solved – Stepwise regression vs. elastic net

Solved – Elastic net arbitrary alpha selection

Related Question