We use regularized Linear Regression to prevent the model from overfitting (reduce model complexity).
Does the same idea hold with regularized Logistic Regression?
Is regularized Logistic Regression a solution to the problem of separation? if yes, how?
I am sure I had some misunderstanding, can anyone help to clarify that for me.
Best Answer
Yes. The bias-variance trade-off exists in all areas of statistics.
Yes; even a small penalty on the coefficients will bound them away from infinity. This is because you will not be able to improve model fit to be arbitrarily good without eventually trading-off with an increase in penalty on coefficients.
See also: How to deal with perfect separation in logistic regression?