Solved – Specifying a Laplace prior using a Gaussian random variable with Gamma variance

gamma distributionlaplace-distributionprior

I need to place a Laplace prior on a random variable, however, I want to use a Gaussian distribution whose variance is Gamma(1,1) distributed, i.e.,

\begin{align}
x &\sim N(\mu,\sigma^2)\\
\sigma^2&\sim\text{Gamma}(1,1)
\end{align}

My question is: Given this above setup, what is the corresponding form of my Laplace distribution (location and scale) for this Laplace prior?

Best Answer

So the question seems to have stemmed from a discussion on this forum: http://social.microsoft.com/Forums/en-US/47b613d0-177d-4ce3-b54d-2476144ece6b/double-exponential-prior-migrated-from-communityresearchmicrosoftcom?forum=infer.net

The general idea, that the forum alludes to, is that you can build a distribution that resembles a Laplace distribution from a Normal distribution by specifying the variance of the Normal distribution to follow a Gamma distribution, i.e.,

\begin{align} x|\sigma^2 &\sim N(\mu,\sigma^2)\\ \sigma^2&\sim\text{Gamma}(1,1) \end{align}

However, as the OP asks, how to specify the location ($\mu$) and scale ($b$) of the corresponding Laplace distribution is not obvious. And in fact, you can't since the distribution will not be exactly a Laplace distribution, but just something very similar.

To better understand the question we (below) a plot of many Laplace distibutions with differing locations ($\mu$) and scales ($b$).

enter image description here

And so generating random variables from Normal distribution with variance following a Gamma(1,1) distribution will give us something that looks very close to a Laplace distribution.

enter image description here

However, it is only an approximation and is not exact.


On the other hand, there is a way to generate a Laplace distribution (with known location ($\mu$) and scale ($b$)) by using a Normal distribution with variance corresponding to a Rayleigh distribution.

The result is the following:

If $X|Y \sim N(\mu,\sigma=Y)$ with $Y \sim \text{Rayleigh}(b)$ then $X \sim \text{Laplace}(\mu, b)$.

Related Question