Background:
I had to perform a data analysis for a client (some kind of lawyer) who was an absolute beginner in statistics. He asked me what the term "statistical significance" means and I really tried to explain it… but since I'm not good at explaining things I failed 😉
Solved – How would you explain statistical significance to people with no statistical background
communicationinferencestatistical significance
Related Solutions
fwiw the medium length version I usually give goes like this:
You want to ask a question of a population but you can't. So you take a sample and ask the question of it instead. Now, how confident you should be that the sample answer is close to the population answer obviously depends on the structure of population. One way you might learn about this is to take samples from the population again and again, ask them the question, and see how variable the sample answers tended to be. Since this isn't possible you can either make some assumptions about the shape of the population, or you can use the information in the sample you actually have to learn about it.
Imagine you decide to make assumptions, e.g. that it is Normal, or Bernoulli or some other convenient fiction. Following the previous strategy you could again learn about how much the answer to your question when asked of a sample might vary depending on which particular sample you happened to get by repeatedly generating samples of the same size as the one you have and asking them the same question. That would be straightforward to the extent that you chose computationally convenient assumptions. (Indeed particularly convenient assumptions plus non-trivial math may allow you to bypass the sampling part altogether, but we will deliberately ignore that here.)
This seems like a good idea provided you are happy to make the assumptions. Imagine you are not. An alternative is to take the sample you have and sample from it instead. You can do this because the sample you have is also a population, just a very small discrete one; it looks like the histogram of your data. Sampling 'with replacement' is just a convenient way to treat the sample like it's a population and to sample from it in a way that reflects its shape.
This is a reasonable thing to do because not only is the sample you have the best, indeed the only information you have about what the population actually looks like, but also because most samples will, if they're randomly chosen, look quite like the population they came from. Consequently it is likely that yours does too.
For intuition it is important to think about how you could learn about variability by aggregating sampled information that is generated in various ways and on various assumptions. Completely ignoring the possibility of closed form mathematical solutions is important to get clear about this.
If the audience really has no statistical background, I think I would try to simplify the explanation quite a bit more. First, I would draw a coordinate plane on the board with a line on it, like so:
Everyone at your talk will be familiar with the equation for a simple line, $\ y = mx + b $, because that's something that is learned in grade school. So I would display that alongside the drawing. However, I would write it backwards, like so:
$\ mx + b = y $
I would say that this equation is an example of a simple linear regression. I would then explain how you (or a computer) could fit such an equation to a scatter plot of data points, like the one shown in this image:
I would say that here, we are using the age of the organism that we are studying to predict how big it is, and that the resultant linear regression equation that we get (shown on the image) can be used to predict how big an organism is if we know its age.
Returning to our general equation $\ mx + b = y $, I would say that x's are variables that can predict the y's, so we call them predictors. The y's are commonly called responses.
Then I would explain again that this was an example of a simple linear regression equation, and that there are actually more complicated varieties. For example, in a variety called logistic regression, the y's are only allowed to be 1's or 0's. One might want to use this type of model if you are trying to predict a "yes" or "no" answer, like whether or not someone has a disease. Another special variety is something called Poisson regression, which is used to analyse "count" or "event" data (I wouldn't delve further into this unless really necessary).
I would then explain that linear regression, logistic regression, and Poisson regression are really all special examples of a more general method, something called a "generalized linear model". The great thing about "generalized linear models" is that they allow us to use "response" data that can take any value (like how big an organism is in linear regression), take only 1's or 0's (like whether or not someone has a disease in logistic regression), or take discrete counts (like number of events in Poisson regression).
I would then say that in these types of equations, the x's (predictors) are connected to the y's (responses) via something that statisticians call a "link function". We use these "link functions" in the instances in which the x's are not related to the y's in a linear manner.
Anyway, those are my two cents on the issue! Maybe my proposed explanation sounds a bit hokey and dumb, but if the purpose of this exercise is just to get the "gist" across to the audience, perhaps an explanation like this isn't too bad. I think it's important that the concept be explained in an intuitive way and that you avoid throwing around words like "random component", "systematic component", "link function", "deterministic", "logit function", etc. If you're talking to people who truly have no statistical background, like a typical biologist or physician, their eyes are just going to glaze over at hearing those words. They don't know what a probability distribution is, they've never heard of a link function, and they don't know what a "logit" function is, etc.
In your explanation to a non-statistical audience, I would also focus on when to use what variety of model. I might talk about how many predictors you are allowed to include on the left hand side of the equation (I've heard rules of thumb like no more than your sample size divided by ten). It would also be nice to include an example spread sheet with data and explain to the audience how to use a statistical software package to generate a model. I would then go through the output of that model step by step and try to explain what all the different letters and numbers mean. Biologists are clueless about this stuff and are more interested in learning what test to use when rather than actually gaining an understanding of the math behind the GUI of SPSS!
I would appreciate any comments or suggestions regarding my proposed explanation, particularly if anyone notes errors or thinks of a better way to explain it!
Best Answer
Differences happen as a result of chance.
When we believe something is statistically significant we believe the difference is larger than can reasonably be explained as a chance occurrence.