[Math] Kullback-Leibler divergence of two Laplace distributions with different parameters

Question

Let the Kullback-Leibler (KL) divergence of two distributions $p(x)$ and $q(x)$ be defined as

$D(P||Q) = E_p(\log p(x) – \log q(x))$

Let

$p(x) \sim \text{Laplace}(\mu_1, b_1)$

and

$q(x) \sim \text{Laplace}(\mu_2, b_2)$

then

$
\begin{align}
D(P||Q) &= E_p(\log p(x) – \log q(x)) \\
&= E_p\left( \log \frac{b_2}{b_1} – \frac{|x-\mu_1|}{b_1} + \frac{|x-\mu_2|}{b_2}\right)\\
&= \log \frac{b_2}{b_1}
– \frac{1}{b_1}E_p|x-\mu_1|
+ \frac{1}{b_2}E_p|x-\mu_2|\\
\end{align}
$

The first term is a constant.

The second term is the median divided by $b_1$ (IM NOT SURE ABOUT THIS STEP)

The third term is given by $b_1/b_2$, because if

$X\sim\text{Laplace}(\mu_1,b_1)$

then

$X-c\sim\text{Laplace}(\mu_1-c,b_1)$

from which follows that

$|X-c|\sim\text{exp}(b_1^{-1})$

Hence the Kullback-Leibler divergence of two Laplace distributions is given by

$
\begin{align}
D(P||Q) &= E_p(\log p(x) – \log q(x)) \\
&= \log \frac{b_2}{b_1}
– 1
+ \frac{b_1}{b_2}
\end{align}
$

I couldnt find the correct answer anywhere online so I was wondering if anyone here knows the right answer.

Answer 1

For the unsure term, you can apply the same reasoning as for the third to get that it is the expectation of an exponential distribution with parameter $b_2^{-1}$ and divide that mean by $b_2$.