[Math] poisson limit theorem for multinomial distribution

approximationprobability distributionsprobability theoryprobability-limit-theorems

It is well known that the poisson Distribution may be used as an Approximation to the binomial distribution, under certain conditions. But now let $X = (X_{1},…,X_{k})$ be multinomial distributed with index $n$ and Parameter $\pi = (\pi_{1},…,\pi_{n})$ with $\pi_{1}+…+\pi_{n} = 1$, i.e. each $X_{i}$ are Independent and binomial distributed with Parameters $n$ and $\pi_{i}$ and $X_{1} + … + X_{k} = n$. Is it also true that
\begin{equation}
\mathbb{P}(X_{1}=n_{1},…,X_{k}=n_{k}) \xrightarrow{ n \to \infty } \prod_{j=1}^{k}e^{-\sigma_{j}}\frac{\sigma_{j}^{n_{j}}}{n_{j}!}
\end{equation}
where $n\cdot \pi_{j} \rightarrow \sigma_{j}$ for $n \to \infty$.

Now we know that each $X_{i}$ is asymptotically poisson distributed to the paramater $\sigma_{i}$. Since they are Independent can I conclude that the Limit random variables are also Independent? If so then I can conclude that

\begin{equation}
\mathbb{P}(X_{1}=n_{1},…,X_{k}=n_{k}) \xrightarrow{ n \to \infty } \mathbb{P}(Y_{1}=n_{1},…,Y_{k}=n_{k}) = \prod_{j=1}^{k}e^{-\sigma_{j}}\frac{\sigma_{j}^{n_{j}}}{n_{j}!},
\end{equation}
where each $Y_{i}$ is poisson distributed to the Parameter $\sigma_{i}$.

Best Answer

An example of something along these lines that would make sense is the following. Fix a function $f : \mathbb{N} \to \mathbb{R}_{++}$ and a positive integer $q$. For each $n=kq$ for $k \in \mathbb{N}$, consider the multinomial distribution on $n$ objects distributed into $qn$ bins, where the probability associated to the $i$th bin is proportional to $f(i)$ i.e. they are $\frac{f(i)}{\sum_{j=1}^{qn} f(j)}$. Then as $k \to \infty$ you might hope to see a "multivariate Poisson distribution" under suitable assumptions on $f$ (e.g. its range is contained in some $[a,b]$ with $0<a<b<\infty$). Moreover you might hope to see asymptotic independence in the sense that you describe, because any one bin asymptotically cannot affect the other bins very much.

But for a fixed number of bins this is a total non-starter: the dependence between the bins will remain significant no matter how big $n$ is. Moreover the occupation number of at least one bin must be diverging in probability; it cannot be that all of the occupation probabilities are simultaneously $O(1/n)$ unless the number of bins goes to infinity.

What I described in the first paragraph is a kind of "thermodynamic limit" in which you simultaneously send the number of objects and the "space" that they are allowed to occupy to become large, at asymptotically the same "rate". You might look into statistical mechanics if you are interested in this type of problem.

A different way to make sense of this is to embrace the idea that at least one occupation number is diverging in probability. One way to do that is to say that a Bin($n,\lambda/n$) random variable describes the sum of the occupation numbers of the other bins. Then the components contributing to the low occupation number bins can become asymptotically independent Poissons with parameters summing up to $\lambda$.

Related Question