Simulation – Using Computer Simulations to Understand Graduate-Level Statistical Concepts

computational-statisticshypothesis testingrsassimulation

Hi I'm taking a graduate course in Statistics and we've been covering Test statistics, and other concepts.

However, I am often able to apply the formulas and develop a sort-of intuition on how stuff works but I am often left with a feeling that perhaps if I backed up my study with simulated experiments I will develop better intuition into the problems at hand.

So, I have been thinking of writing simple simulations to better
understand some of the concepts we discuss in class. Now I could use say Java to:

  1. Produce a random population with a normal mean and standard deviation.
  2. Then take a small sample and try to try to empirically calculate Type-I and Type-II errors.

Now the questions I have are:

  1. Is this a legitimate approach to develop intuition?
  2. Is there software to do this (SAS?, R?)
  3. is this a discipline in Statistics that deals with such programming: experimental statistics?, computational statistics? simulation?

Best Answer

I like your question but don't have specific answers to 2 and 3? I imagine that software packages like SAS (broadly speaking of SAS products and not just SAS/STAT) may have tools that facilitate simulation but I can't say for certain. I don't think this sort of thing fits as a branch of mathematics or statistics.

Now question 1 is what I would like to focus on. Simulation can help in learning statistics at all levels and can aid in statistical research in general. Indeed there are journals focussed on simulation and computation. Even the FDA is recognizing the imprtance of simulation in designing clinical trials and to help predict outcomes.

In the 1960s Julian Simon taught introductory statistics using simulation as a motivator. Although controversial he later claimed that he was doing resampling (permutation and bootstrap) prior to Efron. He published a book using these ideas in 1969. It certainly lacked the theory and was only a teaching aid and not a new approach to statistical estimation. He did not develop any of the mathematical properties that came with and after Efron.

I think for introductory statistics it is useful to do simulation to demonstrate sampling distributions, show how the central limit theorem comes about and physical simulation through the quincunx demonstrates the DeMoivre - Laplace version of the central limit theorem.

Sometimes it enhances intuition. I think that the Monty Hall problem is puzzling and seemingly paradoxical even to mathematicians like Paul Erdos. But simulating the game is often very convincing. There are many problem in probability that are counterintuitive and simulation can, I think help.

In 1978 when I was working on my PhD in extreme value theory I had an intuitive idea for a limit theorem that I was trying to prove. I struggled with the mathematics. Then I decided to simulate the stochastic process and the simulation "confirmed" my result. This gave me the confidence to push on a prove it.

So even at the graduate level and beyond simulation can be useful in two ways.

  1. To help develop intuition as you suggestion in question 1 but also

  2. To confirm intuition as I did in my thesis

Related Question