Solved – Generating random samples from a given distribution

mathematical-statisticsrandom-generationsimulation

What are the different methods by which we can generate random samples from the existing distribution which are computationally efficient?

One of the common methods given in statistics text is by applying a transformation called the inversion method. We start with a Uniform(0,1) distribution from which we can pick an arbitrary number, and then use the inverse of the CDF of the distribution (its quantile function) to find the member of the desired distribution corresponding to the randomly chosen number between 0 and 1.

What other methods exist for doing the same?

Best Answer

Other methods include accept-reject methods and Markov Chain Monte Carlo (MCMC) methods. Note that the method described in the question is part of the so-called inverse transformation methods. There exists other ad-hoc simulation methods for specific distributions (e.g., the Box-Muller transform for univariate normal distribution).

Related Solutions

Multivariate Normal Distribution – Generating Values from It

If $X \sim \mathcal{N}(0,I)$ is a column vector of standard normal RV's, then if you set $Y = L X$, the covariance of $Y$ is $L L^T$.

I think the problem you're having may arise from the fact that matlab's mvnrnd function returns row vectors as samples, even if you specify the mean as a column vector. e.g.,

 > size(mvnrnd(ones(10,1),eye(10))  
 > ans =
 >      1    10

And note that transforming a row vector gives you the opposite formula. if $X$ is a row vector, then $Z = X L^T$ is also a row vector, so $Z^T = L X^T$ is a column vector, and the covariance of $Z^T$ can be written $E[Z^TZ] = LL^T$.

Based on what you wrote though, the Wikipedia formula is correct: if $\Phi^{-1}(U)$ were a row vector returned by matlab, you can't left-multiply it by $L^T$. (But right-multiplying by $L^T$ would give you a sample with the same covariance of $LL^T$).

Solved – Inverse transformation sampling for mixture distribution of two normal distributions

The "two uniforms" are not absolutely necessary when generating from a mixture, but they make the simulation easy to understand. The mixture of normal distributions, $$rf_a(x)+(1-r)f_b(x)$$ has a probability mass of $r$ associated with the first normal and $(1-r)$ with the second normal. This means that the distribution of $X\sim f$ can be decomposed as $$\mathbb{P}(X\in\mathcal{A})=r\mathbb{P}(X_a\in\mathcal{A})+(1-r)\mathbb{P}(X_b\in\mathcal{A})$$ for any measurable set $\mathcal{A}$, where $X_a$ and $X_b$ are normal random variables with means $a$ and $b$ respectively. This can be reinterpreted as $$X=\begin{cases} X_a &\text{with probability $r$}\\ X_b &\text{with probability $1-r$}\end{cases}$$ meaning that to generate from the mixture, one can follow the steps

Pick between components $a$ and $b$ by generating a uniform $U\sim\mathcal{U}(0,1)$ and, if $U<r$ take $\mu=a$ and else take $\mu=b$;
Generate $X$ as $X_a$ or $X_b$ depending on the first step result, by generating a uniform $V\sim\mathcal{U}(0,1)$ and take $X=\Phi^{-1}(V)+\mu$

This explains for the use of two uniforms.

Best Answer

Related Solutions

Multivariate Normal Distribution – Generating Values from It

Solved – Inverse transformation sampling for mixture distribution of two normal distributions

Related Question