Solved – Number of samples needed in Monte Carlo simulation: how good is this approximation

monte carlosimulation

In Risk Theory Beard, Pentikanen and Pesonen (1969) mention a method of assessing number of samples needed for Monte Carlo simulation as

$$ \sigma = \sqrt{\frac{p(1-p)}{s}} \leq \frac{1}{2} \sqrt{ \frac{1}{s}} $$

where $F(x) = p$, i.e. it is a probability of observing some value $x$ and $s$ is a number of samples. This shows us that with 99% confidence value we can expect that values observed in simulation study will lie $\pm 2.576 \sigma$ from $p$'s. This is similar to simulation standard error estimation based on observed variance mentioned by Aksakal. The authors seem to suggest that the formula can be used before the simulation to assess number of samples needed ($s$) to obtain simulated results with some precision of interest.

How good is this approximation?

Best Answer

The approximation could be poor when $p$ is close to zero or one, but when $p = 1/2$ it holds exactly.

The idea here is that we want to estimate the probability of an event by using a sample proportion across many Monte Carlo trials, and we want to know how accurate of an estimate that proportion is of the true probability. The standard deviation $\sigma$ of a sample proportion is as the authors note $\sqrt{p (1 - p) / s}$ (where $s$ is the number of Monte Carlo simulations), but the problem is we don't know $p$. However, we can maximize $\sigma$ with respect to $p$ and get a conservative "estimate" of this standard error that will always hold no matter what $p$ happens to be. This may end up causing us to run more simulations than we need to, but this won't matter as long as the iterations themselves are computationally cheap.

Related Question