Solved – Logistic Regression is a nonlinear regression problem

logisticregressionself-study

I come to a statement that logistic regression is a non linear problem. How can one show this?

Is it possible to treat logistic discrimination in terms of equivalent linear regression problem?

Best Answer

Recall that the Logistic regression model is a non linear transformation of $\beta^Tx$

  • Probability of $(Y = 1)$: $p = \frac{e^{\alpha + \beta_1x_1 + \beta_2 x_2}}{1 + e^{ \alpha + \beta_1x_1 + \beta_2 x_2}}$
  • Odds of $(Y = 1)$: $ \left( \frac{p}{1-p}\right) = e^{\alpha + \beta_1x_1 + \beta_2 x_2}$
  • Log Odds of $(Y = 1)$: $ \log \left( \frac{p}{1-p}\right) = \alpha + \beta_1x_1 + \beta_2 x_2$

So to answer your question, Logistic regression is indeed non linear in terms of Odds and Probability, however it is linear in terms of Log Odds.


A simple example

Fitting a logistic regression model on the following toy example gives the coefficients $\alpha = -5.05$ and $\beta = 1.3$

Plotting the probability $P(Y=1)$ as a function of $X$ clearly shows the non linear relationship

enter image description here

The Odds of $Y$ being 1 given $X$ is also non linear

enter image description here

Finally the log odds of $Y$ being 1 is a linear relationship

enter image description here

See here for some more details: Calculating confidence intervals for a logistic regression