Solved – Logistic / multinomial regression as two / multiple Poisson regressions

Question

Can we instead of doing logistic or multinomial regression do two or multiple Poisson regressions and then combine Poisson predictions to get probabilistic predictions? If yes, how should we transform data for training?

Answer 1

Assume that you're fitting a multinomial regression model with J categories where the contrast between j and the last category J is modeled as $$ \log \frac{\mu_{ij}}{\mu_{iJ}} = \alpha_j + X_i \beta_j $$ where $X_i$ is a vector of covariates associated with the $i$th case. This is rather a hard optimisation problem because the parameters are coupled by the multinomial's conditioning on $N_i$, the $i$-th case's marginal total.

You can instead fit J separate Poisson regressions with linear predictor of the form $$ \log\, \mu_{ij} = \eta + \theta_i + \alpha_j^* + X_i\beta_j^* $$ where each case gets one nuisance parameter $\theta$.

Subtracting the expression for $\log\, \mu_{ij}$ from $\log\, \mu_{iJ}$ to construct $\log (\mu_{ij}/\mu_{iJ})$ shows that the parameters are related as $\alpha_j = \alpha^*_j − \alpha^*_J$ and $\beta_j = \beta^*_j - \beta^*_J$ (the nuisance parameters and $\eta$ cancel). Thus you get what you came for: fitting multiple Poisson regression models gives you the same result as fitting a single more complicated multinomial regression model.

This exposition is essentially the one in Gérman Rodríguez's always excellent notes, in the section on the 'equivalent log-linear model'.

In general this manoeuvre is called the multinomial-Poisson transformation and was written about by Baker in 1994.

Notice also that you don't need to adjust $X$ at all to use it.