[Math] Likelihood function for logistic regression

logistic regressionregressionstatistics

In logistic regression, the regression coefficients ($\hat{\beta_0}, \hat{\beta_1}$) are calculated via the general method of maximum likelihood. For a simple logistic regression, the maximum likelihood function is given as

$$\ell(\beta_0,\beta_1)=\prod_{i:y_i=1}p(x_i)\prod_{i':y_{i'}=0}(1-p(x_{i'})).$$

What is the maximum likelihood function for 2 predictors? Or 3 predictors?

Best Answer

The same for any finite number of predictors, i.e., $$ p(y_i;\vec{x_{i}})=\frac{1}{1+\exp\{-x_i'\beta\}}, $$ where $x_i=(1, x_{1i},...,x_{pi})$ and $\beta=(\beta_0,...,\beta_p)$. Thus the likelihood function is Binomial $$ \mathcal{L}(\beta)=\prod_{i=1}^m p(y_i;\vec{x_{i}})^{y_i} (1-p(y_i;\vec{x_{i}}))^{1-y_{i}}. $$


E.g., if you have two predictors, i.e., $x_1$ and $x_2$, and total of $n$ observations, i.e., $\{(y_i, x_{1i}, x_{2i}\}$ then the likelihood function is given by $$ \prod_{i=1}^n p(y_i; x_{1i}, x_{2i})^{y_i} (1-p(y_i; x_{1i}, x_{2i}))^{1-y_{i}}, $$ where $$ p(y_i; x_{1i}, x_{2i})=\frac{1}{1+\exp\{-\beta_0 + \beta_1x_{1i}+\beta_2x_{2i}\}}. $$

Related Question