I've made a little questionnaire where participants can rate an answer between 1 and 5. I calculated the mean value, the average value and the standard deviation.
Now I was asking myself if it is possible to calculated a confidence interval for these results and if yes, if this would tell me anything. So I just tested it and used excel to calculate a 95% confidence interval.
Here are the values:
Arithmetic average: 4.60
Median: 5.00
Max: 5.00
Min: 3.00
Standard deviation: 0.63
95% Confidence interval: 0.32
But what is this value telling me? I can be sure by 32% that the values aren't random values? Or is a confidence interval for those kinds of questions useless?
Best Answer
Based on my calculations, it seems that you had about 16 or 17 participants. The typical methods for calculating confidence intervals of means assume that sample means are normally distributed. In the case of very skewed distributions, that assumption is only valid for large samples, which rules of thumb define as at least 20 or 30 (depending on whose thumb you talk to).
Also, your data are ordinal data but not necessarily interval data; the difference between a 3 and a 4 is not necessarily the same as the difference between a 1 and a 2. This also makes the typical methods less valid.
If you want to develop some quantitative measure of variability, I suggest that you use a binomial test to estimate a confidence interval of the median. But I probably wouldn't even do that; the exact numbers aren't particularly meaningful unless you have a random sample and you've tested the questionnaire for validity and reliability or if you're comparing it to a similar question or by some blocking factor.
Considering all of this, I don't trust statistics to be particularly meaningful on individual questions from questionnaires like this. When I run questionnaires like this, I generally just plot histograms. I think they tell you more than the numbers.