I am not a math wizard, so please keep your response simple enough. I need to complete a statistics screening exam for a methods course later on today, and I am hung up on one topic that came up during the practice test. The data set I got was in reference to the number of homicides that have occurred in a number of cities. The range of this data is 0-5. When I am putting together confidence intervals and calculating out as much as two standard deviations from the mean I am getting low values that are negative. Obviously you cannot have a negative number of homicides. When calculating the confidence intervals out to two standard deviations from the mean should I present the low value at ZERO or should I actually present the negative number? For example, if a 95% CI caused the calculation to be -1.5 to 3, would I present that or would I present 0 to 3? Thanks.
Solved – I am getting a number below zero when calculating out two standard deviations from the mean. Is this ok
standard deviation
Related Solutions
The short answer, is no, it is not an error.
As @whuber notes, there is nothing surprising (at least to a statistician) about the fact that two standard deviations below the mean of a count variable could be a negative value. Thus, to answer your question, perhaps it would be more useful to ponder why you might find the result surprising.
Why you might be surprised
- Many introductory statistics textbooks show how you can use the mean, standard deviation, and the normal distribution to make claims like approximately 2.5% of the sample is expected to score below two standard deviations below the mean. You may have generalised this idea to a variable where the assumptions of such a procedure are invalid.
- If you did this, you would be saying to yourself: "this is strange, how is it possible for 2.5% of the data to have counts below -0.6".
Estimating percentiles for counts
- Your variable is not normally distributed, it is a count variable. It is discrete; it is a non-negative integer. Thus, in order to estimate the percentage that is greater than or equal to a given value, you need an approach suited to counts. A basic approach would involve using the sample data to estimate such percentiles. More sophisticated approaches could involve developing a model of the distribution suited to counts, justified by the data and knowledge of the phenomena, and estimated using the sample data.
The answer is edited based on the comment by @Gregor.
1) Standard deviation for frequency of occurrence (FO) is $\sqrt{p(1-p)}$ where p is FO/100 (i.e. the proportion). This holds for large samples (see the figure) as sample size affects the standard deviation (references: 1, 2). Using this equation one can find standard deviation for a range of FOs:
FO sd
0 0
10 0.3
20 0.4
30 0.4582
40 0.4898
50 0.5
60 0.4898
70 0.4582
80 0.4
90 0.3
100 0
The convergence occurs practically at sample sizes > 20:
2) Consequently SDs will be more dependent on the FO than sample size, and does not seem to be a useful metric for frequency of occurrence. Yet, Gregor points out that confidence intervals for proportions, which use variance, are useful. See this link for more information.
Best Answer
It appears unlikely to me that the question would require you to calculate two standard deviations of the data out from the mean - especially given that your data are unlikely to be even symmetric, much less normally distributed (since they are discrete). I see no interesting question that could really be answered by this calculation.
It appears more likely that you are asked to give a confidence interval for the mean. This also involves calculating the standard deviations of the data, but then you calculate the standard error of the mean from this standard deviation by dividing by the square of the sample size and finally construct the confidence interval based on the standard error. This confidence interval is therefore much less likely to go beneath zero (and if it did, you should indeed truncate at zero). Note that the sampling distribution of the mean will be roughly normally distributed as sample size increases, which is why this interval actually answers an interesting question, namely where we expect the actual mean to lie.