Solved – How to calculate predicted values using an lm.ridge object

rridge regression

I've found this line of code to calculate predicted values from a ridge.lm model:

# Predict is not implemented so we need to do it ourselves
y.pred.ridge = scale(data.test[,1:8],center = F, scale = m.ridge$scales)%*% m.ridge$coef[,which.min(m.ridge$GCV)] + m.ridge$ym

Why center is set to FALSE? And why do I need to add the mean of $Y$ to the predicted values?

I thought $X$ values should be scaled before running a ridge regression, which implies the out of samples predictors should be centered and scaled?
And scaling the predictors doesn't imply centering the outcomes of the regression, so why to add the mean of $Y$ to the predictions?

Best Answer

Why center is set to FALSE?

Because of information leakage. The idea with having a testing dataset is to mimic the situation you would have when using your model to make predictions on totally new data. Generally, you would not have the chance to center the data your model will see when it is put to use in a production environment. The correct procedure is to memorize the parameters needed to center the training data, then reuse them on any data you need to make predictions on. I believe the correct call to scale should be:

scale(data.test[,1:8], center = m.ridge$xm, scale = m.ridge$scales)

Which uses the fact that the m.ridge automatically centered and scaled its training data, and memorized the needed scaling parameters in m.ridge$xm and m.ridge$scales.

And why do I need to add the mean of Y to the predicted values?

Its the intercept term in the ridge regression. Because the intercept parameter is unpenalized, the intercept term is the mean of the response, just like in regular linear regression.

Related Question