I found this awesome thread which shows KL divergence between two univariate Gaussians. I was wondering if the same formula worked for KL divergence b/w 2 univariate Poisson distributions.
Or should I use the general KL divergence formula and plug into it the pdf for a Poisson process:
$$\int { pdf1(x)*{ log(pdf1(x)/pdf2(x))} } $$
where for Poisson is $$pdf(x) = (\lambda^x / x!)*e^{-\lambda} $$
Best Answer
This is fairly straight-foward. The log ratio of the densities is equal to:
$$ \log\left (\frac {f_1}{f_2}\right)=x\log\left (\frac {\lambda_1}{\lambda_2}\right)+\lambda_2-\lambda_1$$
Then you take expectation of this expression wrt $ f_1$, which simply replaces $ x $ with its expectation (in this case). So you have:
$$ D_{KL} (f_1||f_2)=\lambda_1\log\left (\frac {\lambda_1}{\lambda_2}\right)+\lambda_2-\lambda_1$$