Solved – Why do we need regularized logistic regression

logisticmachine learningregressionregularization

We use regularized Linear Regression to prevent the model from overfitting (reduce model complexity).

Does the same idea hold with regularized Logistic Regression?

Is regularized Logistic Regression a solution to the problem of separation? if yes, how?

I am sure I had some misunderstanding, can anyone help to clarify that for me.

Best Answer

We use regularized Linear Regression to prevent the model from overfitting (reduce model complexity). Does the same idea hold with regularized Logistic Regression?

Yes. The bias-variance trade-off exists in all areas of statistics.

Is regularized Logistic Regression a solution to the problem of separation?

Yes; even a small penalty on the coefficients will bound them away from infinity. This is because you will not be able to improve model fit to be arbitrarily good without eventually trading-off with an increase in penalty on coefficients.

See also: How to deal with perfect separation in logistic regression?

Related Question